`NULL` and `nil`

The problem

NULL and nil are used in C and Go respectively to represent something’s absence. These are poor tools for the job, because they are values, not types.

Consider the following Go function:

func ProcessConn(c *net.Conn) error {
	val, err := doSomethingWith(c)
	if err != nil {
		return err
	}
	processResults(val)
}

With this function, we receive a pointer to net.Conn, and pass it on to some other function. Do we care if c is nil? In this case, it’s easy to argue “no”. We’re not dereferencing the pointer, so we shouldn’t care whether it’s nil or not. That is probably the job of doSomethingWith:

func doSomethingWith(c *net.Conn) (*Widget, error) {
	if c == nil {
		return nil, fmt.Errorf("We don't have a network connection.")
		// OR
		panic("We don't have a network connection.")
	}
	buf := make([]byte, 1024)
	n, err := c.Read(buf)
	if err != nil {
		return nil, err
	}
	//... process buf
	return w, nil
}

However, if we decide to now do something with c in ProcessConn, do we need to check for nil there?

func ProcessConn(c *net.Conn) error {
	if c == nil {
		return nil, fmt.Errorf("We don't have a network connection.")
		// OR
		panic("We don't have a network connection.")
	}
	addr := c.RemoteAddr()
	err := checkRemoteAddr(addr)
	if err != nil {
		return err
	}
	val, err := doSomethingWith(c)
	if err != nil {
		return err
	}
	processResults(val)
}

And if we do put a nil check there, do we still need one in doSomethingWith? If they’re right next to each other in the same file, and we know everywhere doSomethingWith is called, and that all the callers sanitize the pointer before calling, we can remove the check. If not, we have to leave it if we don’t want to potentially get nil pointer dereferences. If you encounter this situation with functions that are in different files or different packages, it becomes a lot less clear.

Dealing with `NULL` and `nil`

You could argue that getting a nil at this point is a programming error, not an expected exceptional condition. Maybe returning an error isn’t appropriate, and instead panic is the right behavior. Maybe, since it’s a programming error, we shouldn’t bother with checking for nil at all. Maybe the documentation says that nil is not a valid argument, and if someone passes nil into your interface, it’s their fault when it panics. nil is used as a signal sometimes. In some cases, passing or returning nil is valid and in other cases it is not. There’s no way to tell without reading documentation or reading the code.

This leaves us with a few options, but none of them are very good:

check for nil everywhere.
check for nil some places - determined by inspecting callers/callees.
don’t check for nil - burden is on caller to check contract in documentation.

I’ll go through the issues with all of these:

Checking for nil everywhere is possible, but completely impractical. If every function you write that accepts pointers as arguments needs to check one or more values for nil, it will obfuscate the actual algorithms. Furthermore, it’s very silly. I don’t know that I’ve ever seen this done because it’s not a serious solution.
This is the most difficult option. This requires every function that passes a pointer to know details of the internals of the callee (does it check for nil?), and every function that accepts a pointer to know if the caller is passing a sanitized pointer. This responsibility has to go both directions when writing writing a new function. We have to consider what functions will call the new function, and what functions the new function will call, whether to check for nil, and if there are multiple pointer arguments, which to check and which not to check.

This is the easiest option, and there’s something to be said for it. In the earlier example, I could have just done this:

// ProcessConn processes a network connection. The argument c must not be nil.
func ProcessConn(c *net.Conn) error {
	val, err := doSomethingWith(c)
	if err != nil {
		return err
	}
	processResults(val)
}


func doSomethingWith(c *net.Conn) (*Widget, error) {
	buf := make([]byte, 1024)
	n, err := c.Read(buf)
	if err != nil {
		return nil, err
	}
	//... process buf
	return w, nil
}

Notice the comment at the top of ProcessConn. Now we’ve absolved ourselves of responsibility. The burden is now on the consumer of our package to make sure they’re passing valid arguments.

A real solution

In order to find a real solution, we should try and identify the root cause of all of the issues, both the original issue and the problems with the solutions, the real problem.

The real problem, which is hinted at in the first section, is that nil is a value, not a type. Even if you execute one of the above three solutions perfectly, it’s still not ideal. None of the above solutions tell you at compile time whether something is correct or not. You have to wait until runtime to either get an error or a panic. The compiler hapily lets you use a value that may be nil as if it’s not, and even if you check a value for nil, that information is quickly lost - other functions can’t know that you’ve checked it for nil and that it’s safe or unsafe to use.

A function signature in C or Go has no way of saying “I want a valid pointer to a value of this type”, or “I want a valid pointer to a value of this type or alternatively, nothing”, and this is the real problem. This is the reason programmers writing these languages need to read documentation and look at the code of functions they’re calling or being called by to find out if pointers might be nil. Isn’t that what a type system is for? The contract between a caller and callee should be in the callee’s signature. We shouldn’t have to read documentation or inspect a function’s code to know whether passing a nil value is valid or if it will cause a crash.

Some other languages have bolted on some support for nullable or non nullable arguments, but I dont’ think that’s a savory solution. Instead, Boson will just eliminate the concept of nil. This is the real solution.

Pointers are used for two primary things:

sharing a reference to an object.
representing a value which may or may not exist.

Both of those things can be done without nil.

Considerations

Boson is far from the first language to eliminate the concept of nil, but it will nonetheless have big consequences for the language. We can’t just eliminate nil without considering what we lose, and what other considerations have to be made:

Zero Value

One consideration is that pointers are types in and of themselves, and we can declare variables with a pointer type:

var p *int

What should the zero value be for a pointer object, if not nil?

One solution is to make sure a pointer value is never used before it is assigned. I think this is the easiest, cleanest option:

var p *int
doSomethingWith(p)   <-- compile error: "Cannot use uninitialized pointer p"

var p *int
var x int = 5
p = &x
doSomethingWith(p)  <-- This is ok. p has been initialized.

Representing Absence

Since we are eliminating nil, we need some way to represent a value that is unavailable.

Consider a slightly contrived hypothetical cache in Go:

func (c *Cache) Lookup(key string) *Widget {
	if c.Contains(key) {
		return c.Get(key)
	}
	return nil	
}

func doWithCache(c *Cache) {
	w := c.Lookup("mykey")
	if w == nil {
		w = makeWidget("mykey")
	}
	// Use w
	...
}

Instead of nil, we will use Algebraic Data Types to capture this concept.

In Boson, we will have:

func (c *Cache) cacheLookup(key string) Option[*Widget] {
	if c.Contains(key) {
		return Some(c.Get(key))
	}
	return None
}

func doWithCache(c *Cache) {
	var w *Widget
	mw := c.Lookup("mykey")
	match mw {
	case Some(widget):
		w = widget
	case None:
		w = makeWidget("mykey")
	}
	// Use w
	...
}

NULL and nil