As you read in the previous post in this series, we have built a frontline router using Go.
If you cast an eye across our many and varied open source projects, the common theme is that they're generally written in Ruby. We have strong Ruby developers and an excellent track record at building and running Ruby services.
So how did we get to choosing Go?
Back to the beginning
Nick has already explained the problem space well, so time for a little history lesson instead.
Back in the alpha and beta days of GOV.UK, we actually had a router. This router had some of the same characteristics desirable of a frontline piece of infrastructure: it offered good performance and resilience. It was also written in Scala.
There were a few issues with this, which I'll enumerate quickly:
- Scala is great for performance, but quite bad at resource usage
- No-one in the core GOV.UK team had a deep knowledge of Scala, and particularly how the old router worked
- It required us to put the JVM onto our frontline cache boxes (currently just Nginx and Varnish), which adds complexity and resource constraints
Whilst the 1st and 3rd point are things we can deal with, it's number 2 that was the real sticking point. We had to have something we could not only maintain but iterate on and build out as required.
So at this point we've ruled out the old Scala router, and in fact it was removed for launch in place of the Varnish configuration described in Nick's article. In order to move forward with a router, we had to go back to first principles.
Performance
As explained above, while GDS as a whole has projects built on a variety of stacks, GOV.UK is primarily a Ruby shop. The performance characteristics we want from a router (high concurrency, minimal latency added per request, low memory usage so we can contain bursts) mean we need to pick a language with a more aggressive performance baseline. This rules out pretty much all scripting languages.
The router is also, by design, largely an HTTP reverse proxy with some extra statefulness - it has minimal features and complexity. So a strong networking library, or pedigree, is desirable.
New and shiny
Programmers like learning new languages (many spend their spare time working on new things), so giving developers the opportunity to learn new things on work time seems like a good move for many reasons:
- It helps the programming team as a whole. The ideas and methods from other languages help us reason about things better, and allow us to approach old problems in a fresh way
- It helps with retention. Programmers might not feel the need to move elsewhere if they can work on different things at work
- It allows us to try out the features of new languages on tightly scoped projects so we can get a flavour for whether or not they'd be a good fit for larger or more complex things. R&D.
Maintainability
Based on the above criteria of performance, friendliness with HTTP, and wanting to introduce a new language, we have a few strong candidates. Erlang, Go, Scala, and Clojure are all good fits and represent the current state of the art. Since we wanted to use a new language here it makes sense for us to pick something that's easy for all of us to learn and easy for future developers on the team to learn and maintain.
We can immediately rule out Scala based on the previous issues we had with the router. We can also rule out Clojure due to unfamiliarity and us not wanting to deploy the JVM on production. We can now rule out Erlang (as much as it pains me - I personally have a soft spot for Erlang and the actor model of concurrency it gives us) because of the unfamiliar syntax and learning curve.
As this leaves us with just Go in the 'new language' bucket, we can be reasonably sure that it will fit well with our needs. But what of the language itself?
How Go specifically fits our stack
GOV.UK is a collection of loosely coupled applications that speak to each other via HTTP. When we put it like that, it's easy to see why a high performing language which speaks HTTP very well would fit in.
Documentation
Go has wonderful documentation for its standard library. All source code is clearly written, well-commented, and is a great source of learning for new and seasoned Go programmers. The combination of documentation and being open source makes it very easy to compose richer functionality out of modules from the standard library. There's even an excellent guide to writing Go code idiomatically, and a tool that formats your code correctly.
Simplicity
As mentioned above it's fairly easy to compose robust new services that use HTTP just by using the Go standard library. This makes testing and performance a breeze as they're both well tested and very fast - we just need to test our own wrapped functionality (more on that to come).
Ease of deployment
To deploy the Router we just compile a binary and ship that single file to our production servers. This is significantly easier than the rsync/symlink/Unicorn dance we have to perform when deploying Rails services, and it's obviously significantly faster and has less moving parts that could fail.
Concurrency is built-in
When writing high performing network servers it's useful to delegate some of the expensive work to other threads - but threads are difficult to reason about and to manage well. Go handles this using goroutines and other native language constructs that mean we never need to worry about such things. An example of how we use goroutines is in our logger module.
The future
Other colleagues will cover other deeper areas of the router project in more detail soon and explore how we really put it through its paces, but it's a testament to the ease with which they took to Go (a new language for all of us) that we built and deployed the Router in front of the GOV.UK website in just 6 weeks. I look forward to working with it again.
If work like this sounds good for you, take a look at Working for GDS - we're usually in search of talented people to come and join the team.
You can follow Brad on twitter, sign up now for email updates from this blog or subscribe to the feed.