Designing Go Libraries

• Can you tell what the API will do just by looking at the usage, without needing to know the implementation?

• Is there room for misuse?

• How will you test this usage to ensure it’s working as intended?

• Depending on the complexity of the design, exercise its flexibility by sketching out a skeleton for a potential future feature or third-party extension. Is it flexible enough?

• What’s the contract of the API? That is, what does it expect from its inputs, and what guarantees does it provide for its outputs?

type Value struct{ /* ... */ }
  func (*Value) AsBool() bool
  func (*Value) AsFloat64() float64
  func (*Value) AsInt() int
  func (*Value) AsString() string
  func (*Value) Populate(target interface{}) error
  func (*Value) TryAsBool() (_ bool, ok bool)
  func (*Value) TryAsFloat64() (_ float64, ok bool)
  func (*Value) TryAsInt() (_ int, ok bool)
  func (*Value) TryAsString() (_ string, ok bool)

TryAs*

try to convert the value to the corresponding type, returning false if the operation failed.

As*

try to convert the value to the corresponding type, panicking if the operation failed.

Populate

decode the value into a pointer (like json.Unmarshal), returning an error if the operation failed.

type Value struct{ /* ... */ }
  func (*Value) Populate(target interface{}) error

3.2.1. Internal packages

Any package you expose in your library is part of its permanent contract. A tool you can use to reduce the surface area of your library is internal packages.

Go packages inside an 'internal' directory can be imported from the directory’s direct parent and the parent’s other descendants. This allows other components in your library to import these packages, but hides them from users of your library.

Use internal packages liberally to control the surface area of your library. Place packages that are not part of your library’s core responsibilities, helpers and utilities written to aid in implementing the library, packages created just for better code organization but not meant to be part of the public API, all into an internal directory.

With a sufficient number of users [..] all observable behaviors [..] will be depended on by somebody.

Reduce input range

Validate input at entry points and be restrictive about what you accept. You can always loosen restrictions later.

Guard against unexpected mutation

If an API accepts or returns a slice or a map, consider what happens if the caller changes the collection afterwards. Is the slice referencing internal state for an object? You can guard against issues from this by cloning the slice or map at the API boundary. Obviously, be careful about where you use this and how it affects performance.

Return zero values with errors

When an operation fails and you need to return an error and a value, instead of returning a partial value (e.g. return x, err), return the zero value of that type (e.g. return nil, err) — except when partial results are part of the contract of the function, as with Writer.Write.

Think about ordering

When returning slices, consider the order of items in the slice. Is the order part of the contract of that result? Or is it just determined by some internal state, changing which could break someone? In these cases, consider sorting the slice at the API boundary.

type Source interface {
  io.Reader

  Name() string
}

var _ Source = (*os.File)(nil) (1)

func New(src Source) (*Parser, error) {
  // ...
}

1. This verifies that os.File implements the interface at compile time. See Verify Interface Compliance.

Any change to an exported interface is a breaking change.

Adding a method

breaks implementations of that interface that aren’t part of your library — see Adding a method to an interface is a breaking change for an example

Removing a method

breaks functions that consume the interface and call that method

Changing a method signature

breaks both, external implementations of the interface and callers of the method

type Source interface {
  io.Reader

  Name() string
}

func New(src Source) (*Parser, error) {
  // ...
}

 type Source interface {
   io.Reader

   Name() string
+  Offset() int64 // bad: breaking change
 }

Add a new interface

type OffsetSource interface {
  Source

  Offset() int64
}

Upcast to the new interface

func New(src Source) *Parser {
  osrc, ok := src.(OffsetSource)
  if !ok {
    osrc = &nopOffsetSource{src} (1)
  }

  return &Parser{
    osrc: osrc,
    // ...
  }
}

type nopOffsetSource struct{ Source }

func (*nopOffsetSource) Offset() int64 {
  return 0
}

1. If the provided source is not an OffsetSource, wrap it with a type that implements the interface around a regular Source. This implementation just returns zero as the offset for all requests.

type StringWriter interface {
  WriteString(s string) (n int, err error)
}


func WriteString(w Writer, s string) (n int, err error) {
  if sw, ok := w.(StringWriter); ok { (1)
    return sw.WriteString(s) (2)
  }
  return w.Write([]byte(s)) (3)
}

1. Attempt to upcast to io.StringWriter.

2. Use the implementation’s WriteString method, if available.

3. Fallback behavior: Convert the string to a byte slice and write that.

 type Config struct {
   URL string
+  Log *zap.Logger // optional (1)
 }

 func New(c *Config) *Client {
+  log := c.Log
+  if log == nil { (2)
+    log = zap.NewNop()
+  }
   // ...
 }

1. Add a new field to the struct.

2. Gracefully handle absence of the field.

package db

type Option /* ... */ (1)

func Connect(addr string, opts ...Option) (*Connection, error) (2)

1. Option is an opaque type. We’ll skip the implementation for now.

2. A function that accepts a variadic number of these.

func WithTimeout(time.Duration) Option { /* ... */ }

func WithCache() Option { /* ... */ }

db.Connect(addr)
db.Connect(addr, db.WithTimeout(time.Second))
db.Connect(addr, db.WithCache())
db.Connect(addr,
  db.WithTimeout(time.Second),
  db.WithCache(),
)

• It allows us to grow without limits: we can add any number of new optional parameters without breaking existing callers.

• Similar to Parameter objects, it improves readability and reduces risk of misuse when there are multiple parameters with similar types.

• With the Option type being opaque, it keeps the surface area small — we get full control over all possible values of Option.

3.11.1. How to implement functional options

There are multiple ways to implement functional options. This section discusses the version I recommend.

Declare a private struct to hold the final state for options. This struct will get a field for every user-facing option.

type connectOptions struct {
  timeout time.Duration
  cache   bool
}

Declare the Option type, and include an unexported apply method inside it that operates on a pointer of this struct.

type Option interface {
  apply(*connectOptions) (1)
}

For each user-facing option, declare:

From the function, return an instance of this new type. Record parameters of the option (if any) on this instance.

type timeoutOption struct{ d time.Duration } (1)

func WithTimeout(d time.Duration) Option {
  return timeoutOption{d: d} (2)
}

In the apply method of this type, set the corresponding field on the full options struct you declared earlier.

func (t timeoutOption) apply(o *connectOptions) {
  o.timeout = t.d (1)
}

Finally, at the top of Connect, declare a connectOptions variable with your chosen default values, and apply each provided Option to it:

func Connect(addr string, os ...Option) (*Connection, error) {
  opts := connectOptions{
    timeout: time.Second, (1)
  }
  for _, o := range os {
    o.apply(&opts) (2)
  }

  // ... (3)
}

3.11.2. Planning for functional options

You might find yourself designing an API that currently does not have any functional options but you foresee needing them in the future. Unfortunately, you cannot add a variadic Option parameter to a function after publishing it — that’s a breaking change.

 func Connect(
   addr string,
+  opts ...Option, // bad: breaking (1)
 ) *Connection

You can plan for this expansion by defining an Option interface without any implementations.

// Option configures the behavior of Connect.
//
// There are no options at this time.
type Option interface {
  unimplemented() (1)
}

func Connect(addr string, opts ..Option) *Connection

3.11.3. When to use functional options

Functional options are very flexible, but you shouldn’t stick them everywhere.

3.11.4. Parameter objects vs functional options

Functional options and Parameter objects solve a similar problem: adding new optional parameters to a function. Picking between the two depends on API-specific needs, but the following can help reason about it.

db.Connect(addr)

db.Connect(db.Config{Addr: addr})

db.Connect(addr,
  db.WithTimeout(time.Second),
  db.WithCache(),
  db.WithTransport(transport),
  db.WithLogger(logger),
  db.WithMetrics(metrics),
)

db.Connect(db.Config{
  Addr:      addr,
  Timeout:   time.Second,
  Cache:     true,
  Transport: transport,
  Logger:    logger,
  Metrics:   metrics,
})

In general, if the function has two or fewer required parameters, and passing in too many options is rare, functional options may be a good fit. Otherwise, parameter objects are likely a better choice.

Either way, don’t mix the two in the same function.

type UpsertResponse struct { (1)
  Entries  []*Entry
}

func (c *Client) Upsert(ctx context.Context, req *UpsertRequest) (*UpsertResponse, error) { (2)
  // ...
  return &UpsertResponse{
    Entries:  entries,
  }, nil
}

1. UpsertResponse holds the values returned by Client.Upsert.

2. Client.Upsert returns only UpsertResponse and an error.

 type UpsertResponse struct {
   Entries  []*Entry
+  NewCount int
 }

 func (c *Client) Upsert(...) (*UpsertResponse, error) {
   // ...
   return &UpsertResponse{
     Entries:  entries,
+    NewCount: len(newEntries),
   }, nil
 }

• Do not log and return errors. This causes log statements to be duplicated because the caller will likely log them too. Either log the error and don’t return it, or return the error and don’t log it, allowing the caller to choose whether they want to log it.

• Do not export error types. These don’t compose very well — layers above yours now need to know about your custom error type.

• Do not use pkg/errors. It comes with a significant performance penalty because all errors capture a stack trace, even when not necessary.

3.13.1. Error matchers

Error matchers are function that take an error and report whether it’s a specific kind of error.

For example:

func IsNotFound(error) bool

// Elsewhere:

u, err := findUser(...)
if err != nil {
  if !IsNotFound(err) {
    return err
  }
  u = newUser()
}

Use Upcasting to implement this.

type errNotFound struct {
  // ...
}

func IsNotFound(err error) bool {
  _, ok := err.(*errNotFound)
  return ok
}

3.13.2. Error metadata accessors

Error accessors are functions that accept an error and return library-specific metadata associated with them.

For example:

func MissingKeys(error) []string

// Elsewhere:

us, err := findUsers(...)
if err != nil {
  if missing := MissingKeys(err); len(missing) > 0 {
    log.Printf("Skipping missing users: %q", missing)
  } else {
    return err
  }
}

Implement these similarly with Upcasting, and return an fallback value for when upcasting fails.

type errNotFound struct {
  missing []string
}

func MissingKeys(error) []string {
  if e, ok := err.(*errNotFound); ok {
    return e.missing
  }
  return nil
}

Don’t grow unbounded

The number of goroutines you spawn for a request should not be affected by the contents of the request. For example, you can spawn three goroutines for a request, provided that you always spawn no more than three goroutines per request, no matter the contents of the request.

This means that you should not spawn a goroutine to process each item in a slice. If you need to process a dynamic number of items concurrently, use a bounded pool of goroutines.

Don’t leak goroutines

If the work offloaded to goroutines is asynchronous, always provide a means of stopping that goroutine. Your library shouldn’t do work that a user cannot stop when they’re done using your library.

You can use goleak to test for goroutine leaks in your library.

• Effective Go > Names

• What’s in a name?

Write for the right audience

Documentation is intended for users, not maintainers. Don’t list implementation details in the documentation. Tell someone looking at an object’s documentation what they need to know about that object’s purpose, how to use it, and how it connects to other objects.

Don’t bury the lede

Users don’t usually read documentation front-to-back like books. They search for keywords related to information they need, and they start by skimming the documentation to find what they want. Don’t bury key points inside a wall of text. Instead, use paragraphs and lists judiciously to highlight independent pieces of information.

Provide examples

Users will prefer to have code samples that they can adapt to their needs over reading multiple paragraphs of text. Provide them with these. Make heavy use of inline code snippets for small code samples, and Go’s example tests for tested samples.

No non-documenting documentation

Non-documenting documentation is documentation that doesn’t provide any new information. For example:

// RequestHandler handles requests.
type RequestHandler struct{ /* ... */ }

// Handler is an interface for handlers.
type Handler interface{ /* ... */ }

Don’t do this. Instead, explain what the object is, how and why to use it, and anything else the user may need to know.

A commit log does not a changelog make.