Funcs, Interfaces, Generics, and Golang’s Standard Library Problem

Go has a problem in its standard library and to talk about it we’ll first need to discuss high order functions. High order functions is a fancy term for functions which operate on other functions. What exactly does this mean? Consider you’re writing a piece of code which pings a device and reports the status of the device to a server, like so:

type Pinger interface {
  Ping(timeout time.Duration) error
  IPAddr() string
}
func pingDevice(device Pinger, server Server) {
  for {
    result := device.Ping(time.Second)
    server.SendStatus(device.IPAddr(), result == nil)
    time.Sleep(time.Second * 10)
  }
}

Suppose that later down the line you want to re-use this code and instead of reporting the device’s status to a server you want an email to be sent when the device pings down. Instead of copying the code, you make the following adjustment:

type Pinger interface {
  Ping(timeout time.Duration) error
  IPAddr() string
}
func pingDevice(device Device, report func(ip string, result error)) {
  for {
    result := device.Ping(time.Second)
    report(device.IPAddr(), result)
    time.Sleep(time.Second * 10)
  }
}

The report function can now be implemented by sending to a server:

pingDevice(device, server.SendStatus)

Or by sending an email:

pingDevice(device, func(ip string, result error) {
  emailer.SendEmail(
    "operations@acme.com",
    "Critical Alert",
    fmt.Sprintf("Important Device with IP Address %s is offline!", ip),
  )
})

PingDevice is now officially a high order function! There are of course other reasons to utilize higher order functions, but abstraction is a classic example.

Go presents a few different ways to achieve high order dynamic behavior, there are 3 in fact:

1. High order functions (passing functions as variables)
2. Interfaces
3. Generics

While all of the above achieve similar functionalities there is a major divide between func-interface-land and generic-land. Funcs and interfaces operate by selecting different behaviors at runtime while generics operate at compile time. This is an incredibly important fact which has the following consequences:

1. Funcs and interfaces are more dynamic as they can be swapped around at runtime.
2. Generics are (usually) faster because the compiler gets to make better decisions about the path your code takes

When do you need the more dynamic behavior from interfaces/funcs? Let’s say you’re writing a server for a chatroom where user may connect over either the internet within a browser or over SMS. You might write the following interface to represent a chatroom user:

type ChatroomClient interface {
  ReceiveMessage() <-chan Message
  SendUpdate(ChatroomStatus) error
}

We then implement both the SMS and Web versions of ChatroomClient:

type SMSChatroomClient struct {
  IDK
}
type WebChatroomClient struct {
  TODO
}

Finally, the chatroom might look like:

type Chatroom []ChatroomClient

It’s important that the chatroom doesn’t know the concrete type of ChatroomClient so that the Chatroom can easily support both SMS and Web ChatroomClients. If we forced the chatroom to use only the SMSChatroomClient or the WebChatroomClient then our clients couldn’t talk to each other, and that’s sad! Here the dynamic behavior of interfaces helps us allow for dynamic behavior at runtime.

Generic code tends to be more useful when implementing generic utilities. Consider we’re writing a binary tree:

type Tree[T any] struct {
  Data T
  Left *Tree[T]
  Right *Tree[T]
}

While generics are technically less flexible, I argure that this scenario provides higher flexibilitiy with generics as it makes no assumption about the usage of Tree. We don’t know what type T will be, the user may want to use any or just a simple int:

type IntTree = Tree[int]
type AnyTree = Tree[any]

In short, generics allow for compile time behavior selection while interfaces and funcs allow for runtime behavior selection.

Ok, so what’s the problem with Go’s standard library? Well, imagine it’s around 2009 and Go is getting ready to release. One of the last major pieces in the puzzle is writing the standard library itself. Some packages turn out awesome, like io! The interface model makes io.Reader and io.Writer super clean to use, re-use, and test across lots of different programs. But other packages aren’t quite so great, like sort. 99+% of sorting will be done on array/slice types but this package shows little mention of them. Instead, it sorts sort.Interfaces. Given our above discussion, it would seem reasonable to implement a sorting package using generics as it’s mandatory to sort a slice containing items of the same type. There’s only one problem: generics won’t be released for another 13 years. And this introduces a bigger problem into the standard library: features which should be implemented with generics are instead implemented with higher order functions and interfaces.

Even though it’s been a year since generics have been released, they’re hard to find in Go’s standard library. One would expect to find them in packages like sql (I’m talking about you sql.NullType!), sync, container, and sort but I assure you they aren’t! The reason for this, I believe, is twofold:

1. Updating the standard library might cause breaking changes, and this is usually frowned upon from standard library level stuff!
2. If it ain’t broke, don’t fix it.

There are two ways to interpret the lack of generics in the standard library:

1. Generic programming is not a well-accepted way of writing Go code
2. The standard library is no longer the model for writing good Go

And, unfortunately, I think number 2 is true. Generic code is an incredibly useful concept which I’ve found myself using quite frequently since it was released around a year ago, and I believe I’m not alone there. This leaves it up to the community to build better constructs than Go had back in 2009 unless the standard library starts changing quickly!

So what does this mean practically? Well, continuing to use a standard library without generics will result in harder to read and less performant code. As an experiment in the performance gains to be had from migrating to generic code, I’ve copied the standard sort implementation to utilize generic code with a roughly 2x speedup. Generic code also provides mildly stricter type safety over the use of the any type which is always a win in my book. It’s hard not to see a future where many core utility features will move from the standard library into third party libraries.