Eris – A better way to handle, trace, and log errors in Go
github.comHave spent a while in the py/js world but have switched to rust/go for part of this year -- biggest change is boilerplate relating to error handling.
Automatic stack capture for exceptions is something my language could conceivably do on my behalf.
Writing even 3 lines of code per function to propogate up the error is a huge pain, especially because it pollutes the return type -- MustWhatever() in go is much easier to use than Whatever() returning (type, err)
What surprises me as a primarily JVM developer now supporting Go apps and dabbling in Go development is that the logging libraries (like klog) I've encountered don't support the level of configuration I'm used to in JVM apps. I miss being able to configure individual loggers at desired levels with a logback.xml in the JAR, but especially the ability to change logging levels at runtime.
I guess it's part of the Go philosophy - more dynamic logging would most likely incur relatively higher performance costs.
You might wanna look at using something like zap instead, as it enables you to do runtime loglevel changes...
Cheers for my development, I'll do that :)
You're fighting the language. Unless you're writing prototypes where crashing doesn't matter, you should be writing error handling code first, and your business logic second -- so-called "sad path first" programming, c.f. "happy-path first" programming that you usually do in languages with exceptions like Python.
I'm super into the error handling philosophies of Go and Rust, but I don't think either of them is "sad path first". That implies that you can think about your error cases in a meaningful way before you have concrete business logic, which seems unlikely.
Of course you can. Sad path first means, after you write your data types and interface signatures, writing the first stub implementations as
and then filling them in gradually likefunc (t *Thing) Process(id int) (string, error) { return "", fmt.Errorf("not implemented") }
and explicitly notfunc (t *Thing) Process(id int) (string, error) { dat, err := t.store.Read(id) if err != nil { return "", fmt.Errorf("error reading ID: %w", err) } cert, err := dat.ExtractCertificate() if err != nil { return "", fmt.Errorf("error extracting certificate: %w", err) } return cert.Name(), nil }
Writing code this way, explicit error handling upfront, is fundamental to reliability (for a large class of applications).func (t *Thing) Process(id int) (string, error) { dat, _ := t.store.Read(id) // TODO: error handling cert, _ := dat.ExtractCertificate() // TODO: error handling return cert.Name(), nil }Sure, but in a better language, the second version already does the exact same thing as the first one. Your `return fmt.Errorf` is not exception handling, it is simply manual exception bubbling. It is boilerplate that you can forget to add, and that makes it harder to understand what the code is supposed to do. Maybe for the first error you are adding some context that the Read function didn't have, but for the second, the error handling code is doing absolutely nothing that an automatic exception stack trace didn't do.
Even worse, with this style of programming, someone up the stack who would actually want to handle these errors has no mechanism to do, since you're returning the same type from both error cases. If they wanted to handle the certificate error but not the read error, they would have to parse the error message string, which is brittle. But if you did want to add appropriate context, your function would bloat even more. Not to mention that the standard library doesn't really help, since it generally doesn't define any specific error types to set up some patterns for this. Even in your example, from the start you assumed that your function can either succeed or fail in a generic way, you didn't think that the signature may want to encode multiple different error types, which is what GP was talking about when saying you can't usually think about the sad case before the happy case. Sure, if the extent to which you want to define sad case first is 'sad case can happen', you can.
Go's error handling strategy is its weakest aspect, and it is annoying to hear advocates pretend that Go is doing it right, when 90% of Go code is doing the same thing as 90% of exception-based code, just manually and with less helpful context for either logging or for the 10% of code which actually wants to handle errors.
> Sure, but in a better language, the second version already does the exact same thing as the first one. Your `return fmt.Errorf` is not exception handling, it is simply manual exception bubbling.
You look at what I'm doing as a more tedious and error-prone version of exception bubbling, but that misses the forest for the trees. The whole point of doing it this way is to lift errors out of the shadows of the exception control flow path, and put them front-and-center in the actual logic of the application. Programming (in many domains) is error handling, the error handling is at least and arguably more important than the business logic.
I don't want exceptions. I do want this (or something like it).
> Even worse, with this style of programming, someone up the stack who would actually want to handle these errors has no mechanism to do, since you're returning the same type from both error cases.
As the author of this module, I get to decide what my callers are able to see. What I've written is (IMO) the most straightforward and best general-purpose example, where callers can still test for the wrapped errors if they need to. If it were important for callers to distinguish between Read and Certificate errors, I would use sentinel errors e.g. var ErrCert (if the details weren't important) or custom error types e.g. type CertificateError struct (if they were).
Adding this stuff isn't bloat. Again, it's just as important as the business code itself.
> Go's error handling strategy is its weakest aspect, and it is annoying to hear advocates pretend that Go is doing it right
In industry, considering languages an organization can conceivably hire for, and considering the general level of industry programmers -- programs written in Go are consistently in the top tier for reliability, i.e. fewest bugs in logic, and fewest crashes. Certainly more reliable than languages with similar productivity like Python, Ruby, Node, etc.
There are plenty of flaws in Go's approach to error handling -- I would love to have Option or Result types, for example -- but I think, judging by outcomes, it's pretty clear that Go is definitely doing something right.
> programs written in Go are consistently in the top tier for reliability.
Citation needed. It may simply be that those code bases are doing relatively trivial work compared to large programs in other languages, where bugs are more likely to happen simply due to code size. Even in this thread another poster wrote:
> I've accidentally introduced way more bugs through Go style error handling than through Python style error handling.
I've seen production golang code where errors were being silently overwritten. Much, much worse than anything in Java or C# where exceptions are explicitly swallowed.
I'm not missing the forrest for the trees, I know your argument and reject it.
Again, if you had showed an example where something is actually being done with the errors, I would have agreed with you 100%. But when all that is being done is bubbling the errors, having this be done manually by the programmer (and read every time by the code reviewer) is both inefficient and error-prone. Not to mention that one of the first 'skills' I developed as a Go programmer was to ignore any block starting with 'if err != nil', since it appears so, so much in the code. It's not uncommon to have one function contain 10 different 'if err != nil { return nil, fmt.Errorf("Error doing X %v", err)}' for trivial logic (make 10 calls to some external service, abort if anything fails).
I don't have a problem with encoding errors in the function return type. But, coupled with Go's inability to abstract any kind of control flow, this error 'handling' strategy is almost as bad as C's. Other languages that don't offer exceptions avoid this problem with higher level control mechanisms, such as monads or macros.
Even worse, the Go designers recommend some horrible patterns [0], like Scan() not returning an error, but putting the scanner in an error state that all other Scanner functions respect, and having client code explicitly check for the Error() property of the scanner object at the end - preventing any generic tool from helping check whether you correctly handle errors in your code, and introducing an entirely different pattern.
And I don't know the source of your claim about Go's reliability, but all of the studies I have read comparing programming languages have found no or very little effect of the choice of language on overall number of bugs. One recent study [1] which included Go did have it as one of the more reliable languages (but behind Ruby, Perl or Clojure), but with a very minor overall effect, that may be explained by many factors other than error handling (they did not compare languages by this aspect).
Edit: And one minor point, but I did miss the %w in your example code, which does indeed make it possible for code consuming your errors to differentiate them. In my defense, this is a feature of the very newest version of Go only; and having the difference between a 'testable' error and a not testable one be %w vs %v in a format string seems a design decision particularly hostile to code review.
[0] https://blog.golang.org/errors-are-values
[1] https://www.i-programmer.info/news/98-languages/11184-which-...
> I know your argument and reject it.
And I yours, so it seems we're just at an impasse of opinion. That's fine. We can continue being productive in our own ways, and history will judge the superior approach, in whole or part.
`If they wanted to handle the certificate error but not the read error, they would have to parse the error message string, which is brittle.` Aaargh that is my main pain with Go error handling. I try to have an open heart with all the verbose way of living, but this "select by error type" when I want to treat errors really feels awkward
Rust has a postfix `?` operator to propagate errors up the call stack.
as I understand it, this requires an Option or Result return type that matches the type of the statement
The ? operator is better than nothing, but I still need my error types to match all the way up the stack
AFAIK if you properly define the From/To operator, rust generates it for you.
And if you think about it, defining those operator is a good practice as it tells your program how to handle errors.
Or you can use a boxed error iirc. Matching on the type of error after you do that isn't exactly great but it's doable if all you want to do is bubble the error up.
Places where that approach really sucks include web api routes (unless I haven't figured out a trick yet). For example, rocket defines Responder on Result so long as the ok and error variants both define it, but since a boxed error is erased it doesn't function correctly.
You can use the Error trait [1] from the stdlib, implement it for your error types and then just use the trait in your Result. That way you don't need the types to match and you can convert between then as needed while adding the previous errors to the chain.
ah thanks
It doesn’t require it. Instead it could just generate the same error handling boilerplate. But I think it would be a mistake versus general purpose (generic) sum types a la Rust.
I don’t buy the “huge pain” argument. I write lots of Python and Go, and the error boilerplate is a non-issue. I also appreciate that it’s explicit instead of implicit.
It's not just a pain to write. I've accidentally introduced way more bugs through Go style error handling than through Python style error handling. Some examples:
Forgetting that a function returns an error:
Forgetting to check the error returned by a function. Note a linter won't pick this up since the err variable is used later.... foo() // foo returns an error that isn't being handled. ...
Accidentally typing return nil instead of return err:... err := foo() err = bar() // The previous error will go unhandled. ...
And in the case of the errors library, there's times where I will call a builtin function that returns an error and forget to call errors.WithStack. Every once in a while I'll come across an error without a stack trace and I'll have to hunt down where it came from:... if err != nil { return nil } ...
All of these issues look just like normal bug free Go code. On the basis that I've introduced more bugs this way, I prefer Python style error handling by far.... err := json.Unmarshal(bytes, &obj) if err != nil { return err // should be errors.WithStack(err) } ...I don’t know man. I have scarcely seen bugs like these and most of them seem pretty conspicuous to me (maybe I’ve just been writing Go long enough), but I see loooads of uncaught Python (and Java) exceptions internally and from high-profile third party tools.
That's very possible. It's very easy to forget try...catch and wind up with uncaught exceptions.
Honestly though, I would prefer the uncaught exception case. When an uncaught exception is thrown, it's very clear you have an uncaught exception and you know exactly where it came from. In the examples I wrote, you will accidentally catch an error silently. You will never know if anything went wrong unless silently catching the error triggers an issue somewhere else. Even then, it's pretty hard to trace back the bug to the lack of error handling code somewhere else.
Try golangci-lint it catches all your use case.
Thanks! It looks like by default it catches the first two. Is there a way to configure it to catch the other two? I don't think it can catch the case where you accidentally return nil because sometimes you do actually want to return nil when you see an error. I also couldn't find any linter that checks that you are using errors.WithStack when needed.
Linting for WithStack seems like it might be tough since you want to make sure the error was annotated exactly once (I think that’s the intended use, anyway?). The linter would need to know whether or not a fallible function call has annotated the error or not. Seems like an interesting exercise in any case.
Having written maybe two lines of Go, many years ago, I guess I'm surprised that all of these cases make it past the type system. Like shouldn't it bitch if you try to return nil when it's expecting a non-nil value or an error? I guess I'll go try to find an online go thinger to find out.
There’s no way to tell Go that a value shouldn’t be nil.
That's not exactly true. All struct types in Go can not be nil. However, they also can't be abstracted over in any way, so they are a poor idea for an error type - you want an error interface, and all interface types in Go indeed are nil-able.
You can abstract over a struct as easily as a pointer, and structs work fine for satisfying interfaces. Specifically, structs make it a little harder for someone to inadvertently mutate things (passed by copy) and because they can’t be nil you don’t need to worry about a non-nil interface implemented by a nil concrete pointer (this is mostly only a problem for people who are new to pointers in my experience). The downside is every time you put a struct into an interface, it gets allocated on the heap, and allocations are expensive in Go (also passing structs can be more expensive than passing pointers).
What I meant about abstraction is that if you define a function which returns an error struct instead of an error interface, you lose any abstraction capability, you can only return that specific struct (and need to find some way of signaling that no error occurred).
Ah, I see. Yes, that’s true. You would need to return a bool alongside it or similar, but the best practice is to use an interface for sure.
Thanks for the info!
And... Woah. That's kind of horrifying to me (though I totally get the explicitness of it)... does this just mean liberal amounts of: thing != nil everywhere? Or is the rest of the memory management I guess, uhh.., good/magical enough you don't have to worry about it constantly in calls further down the stack if you've checked it once? Or are you always feeding the nil check beast?
> does this just mean liberal amounts of: thing != nil everywhere
Yes, it does. Whenever you call a function that can possibly fail, you are supposed to add:
You can think of it as unwinding the stack by hand. That's why a lot of people complain about Go error handling so much. That and a lack of generics. Looking at some code I've written, about 15-20% of the lines in a file are responsible for error handling.if err != nil { return err }> Or is the rest of the memory management I guess, uhh.., good/magical enough you don't have to worry about it constantly in calls further down the stack if you've checked it once?
I don't quite understand the question here. Are you asking about a performance impact of having nil checks everywhere? If I had to guess, I would think there's a negligible performance impact because the branch predictor will always predict the happy case. As for memory concerns, as long as you are in the happy case and returning nil, no memory needs to be allocated. It's the same reason null doesn't require any memory allocation in other languages.
> Yes, it does. Whenever you call a function that can possibly fail, you are supposed to add: if err != nil { return err }
I don’t think that’s what the parent meant by his question. He was asking if you have to add runtime checks all over to make sure any reference type isn’t nil; no, you don’t—you only add the checks for those for which nil is a valid state in the program (such as errors). If it’s an invalid state, it will panic because it’s an exceptional circumstance—a programmer error. This isn’t a defense of nil; only a clarification about how nil is dealt with. With this context, the rest of his question (the part you didn’t understand) becomes clearer.
> All of these issues look just like normal bug free Go code.
Not to me.
I don't know. They're hard for me to find. How would you pick up on the first and last example? They are possibly correct depending on the exact functions you are calling.
The other two examples, I could maybe understand, but they still look pretty close to normal Go code. In the case where you forget to handle an error, you need to be able to recognize the absence of the error checking bit. It would be one thing if there was extra code that looked wrong, but looking for the absence of code makes it hard to spot.
In the case where you return nil, it looks exactly like a normal early-exit from a function. You need to be able to recognize that the code is not a normal early-exit, and that the three letters "err" were swapped for the three letters "nil".
ok
Go is just a half-baked joke language. Neither generics nor proper error handling nor a proper type system - whoever uses that crap just because it was made by Google is wasting their time. The only thing Golang has been succesful at is proving Google is well-capable of releasing and marketing absolute unadulterated crap. Just use the JVM or .NET Core...
I think a lot of programmers (and languages) confuse errors and exceptions. In most cases, errors should be expected, like an End Of File error or socket disconnect error, they aren't really exceptions. You don't really want a stack trace if an EOF happens, so you? When there really is an exception, why not just use panic and let it bubble up?
Ehh, I'm not convinced the vocabulary there fits. An error can be an exception, if it was unexpected. That's the real distinction: errors are expected and can (optionally) be handled explicitly. Anything unexpected, in your code or within a library function, is an exception and should bubble up / halt execution. But the trouble is that _expectedness_ isn't often a language construct, but more like business logic, and will have different meanings (and presumed severity) between library author and end user.
I think there are a lot of schools of thought on this precisely because it is a difficult problem to solve: it requires getting a bunch of programmers to agree about how the edge cases should be handled, and that's no small feat.
Rust has no exceptions. Still, stack bubbling shows back up, implemented semi-manually by setting up each error type to be automatically cast to a more generic error type and then returned. To me that's an indication that exceptions are "real," as a legitimate pattern in real world programs, and not just an invention of overeager programming language designers.
Panic is hidden and not composable.
Composition: Errors as values means it is easy to write functions to deal with them. Errors as exceptions means invoking special language constructs. And program termination doesn't compose well. Library authors always avoid panics. Generally only applications are good at deciding when to panic and when to catch a panic.
Hidden: how do you know if a function panics? If it returns an error, I see that in the type signature and I can be forced to handle it by linting tools. People seem to not like Javas checked exceptions, but I think that like most things Java it had to do with some of the implementation details, I think the concept is much preferable to hidden exceptions/panics.
What's an error and what's an exception depends significantly on context.
For the socket disconnect for example, maybe a child process maintains a connection to the parent manager process. A socket disconnect is absolutely unexpected then and explicitly handling it adds noise.
My friend and I have been working on eris for a while and we’re very excited to get feedback from fellow gophers. eris provides a better way to handle, trace, and log errors in Go. More info here github.com/rotisserie/eris and godoc.org/github.com/rotisserie/eris
You can reach out to us on slack (gorotisserie.slack.com/archives/CS13EC3T6) for additional comments and feedback. Thanks!
Looks nice! Any plans to add support for uploading stacks to Sentry? That's a must have for us and why we are using pkg/errors.
Thanks! We'd love to add support if you need it. I'll make a note of it but feel free to add an issue for it if you have time!
Stack traces and structured output are handy, but I was hoping for a way to factor out the handler boilerplate that makes app code unreadable (like the "check" or "try" proposal for future versions).
This is really cool! My company has a similar internal library but it doesn't format wrapped errors as well as Eris.
One other issue I've struggled with is in propagating errors across goroutines. If an error is created in the child routine, `runtime.Callers` doesn't include any stack frames from the parent. Assuming the parent wraps the error, it sounds like Eris would give you at least one line of the parent stack trace. Does it handle this specifically by including all of them?
I'd love to see a bit of the docs telling me how this improves on pkg/errors and go 1.13. At first pass, I'm not sure?
It seems to have a bit of a cleaner presentation in the error stack -- Its an array, which feels nicer than a string.
This looks pretty good. And I also get the reason behind it’s name. But for a utility package like this, it’s better if it was named errors. It makes the usage and call sites clear. Ex: eris.New() vs errors.New()
import errors "github.com/morningvera/eris"Yes it can be import aliased everywhere. But my point is it’s not needed.
I wonder if they picked "Eris" based on Greg Hill's Principia Discordia.
I did see that it's named after the Greek goddess Eris.
It's just that she isn't a major goddess, and is famous only for her "apple of beauty" trick. And outside classical scholars, that's arguably via Principia Discordia.
So I'm curious.
IDK about the error json, looks too much like a PHP dump when a descriptive error wrapper will do.
If you are reading the JSON yourself, you’re doing it wrong. JSON logs can be parsed and stored for analysis and generally deal with multi line issues much better than any non json.
Bingo, we emit all our logs as JSON for ingestion into ES to allow easy analysis.