Python errors as values: Comparing useful patterns from Go and Rust

Errors happen in programs -- they're unavoidable! It's important to know both where errors can happen and how to effectively handle them. As we developed our Python SDK, safe and effective error handling was paramount.

In this post we'll:

• Compare the two primary ways of handling errors: thrown errors and errors as values.
• Demonstrate how to handle errors as values in Python (a traditionally thrown-error language).

Thrown errors

A thrown error interrupts the control flow, propagating down the call stack until it's caught. If it isn't caught then the program cashes. A big problem with this approach is a lack of clarity on where an error might occur. For example, which lines in the following code throw an error?

It's impossible to know which line might throw an error without reading the functions themselves... and the functions those functions call... and the functions those functions call. Some thorough engineers may document thrown errors but documentation is untested and therefore untrustworthy. Java is a little better because it forces you to declare uncaught errors in method signatures.

So if we want to be really safe then we'll wrap each call with a try/catch:

As we think about each possible error we realize that our original logic would crash the program when we didn't want it to! But while this is safe it's also extremely verbose. Python engineers overwhelmingly agree, which is why most Python code has 1 large try/catch at best:

So the thrown error approach:

• Doesn't tell us which functions could error.
• Doesn't force us to handle errors where they happen.
• Encourages engineers to use coarse-grained error handling (i.e. one big try/catch).

There has to be a better way 🤔.

Errors as values

Some languages (e.g. Go and Rust) take a different approach: they return errors rather than throwing them. By returning errors, these languages force engineers to notice, think about, and handle errors.

Go returns errors using a tuple (well, not really a tuple but it looks like one!), putting the error last by convention:

Rust returns returns errors using a "wrapper" type called Result. A Result contains both a non-error value (Ok) and an error value (Err):

Regardless of the specific approach, returning errors as values makes us consider all of the places an error could occur. The error scenarios become self-documenting and more thoroughly handled.

Errors as values in Python

So how can we treat errors as values in Python? We could take the Go approach and return a tuple:

Since the type checker doesn't know the tuple values are mutually exclusive, we're forced to do a superfluous assert user is not None. Otherwise, the type checker incorrectly thinks user is nullable.

Next, let's try something Rust-like using the awesome library result, which leverages pattern matching:

Better than the tuple! But we still have some downsides:

• Verbose.
• Language servers don't understand that a return in both cases will always end the function. In other words, they don't know that checking both result.Ok and result.Err is exhaustive.
• Pattern matching is new in Python (3.10). Many people are on older versions and many others are hesitant to introduce the match statement into their codebase.
• External dependency (the result package).

The last approach we'll try is returning a union:

This looks great! We didn't need superfluous assertions (like the tuple approach) and we didn't introduce new patterns (like the result approach). Unions work because isinstance supports type narrowing:

• Within the if isinstance(user, Exception) block, the user variable is narrowed from User | Exception to Exception.
• Since we set user = User() when user is an Exception, type checkers will understand that user cannot be Exception after the if statement.

Conclusion

Inngest's Python SDK handles errors as values, since this integrates error handling into the normal control flow of the program. This makes programs more verbose, but it ensures that we're properly handling errors.

We implemented errors as values using unions, since that was the most idiomatic and terse approach. Other languages use tuples (e.g. Go) or wrapper types (e.g. Rust), but we felt that these patterns either didn't work well in Python, were too verbose, or heavily used a pattern that isn't yet idiomatic.

Build better
agents today