On Go’s Web Application Ecosystem
thechangelog.comWhat I find interesting about Go[1] and the web application community is that people constantly point to speed as a huge selling point. Speed in the sense of latency and request volume per server.
I still feel like for the majority of cases, this type of speed difference is not going to matter. The difference between the Go built in web framework at 190,687[2] (1.3 ms) json responses per second and Flask, a popular Python framework at 18,945[2] (3.1 ms) is trivial when you look at the bigger picture.
If you are receiving enough traffic for that to make a fundamental difference, you'll likely have enough potential or realized revenue to pay for a few more [Flask, Rails, "slow" framework] servers, or have enough developer time to warrant building it in Go.
The upside to using a "slow" framework is massive. Try writing an app like Ticketee[3] in Go, even with Gorilla et al and you'll realize the productivity gains you're missing out on.
For a small API, backend services, command line apps Go makes sense. But the big selling point there isn't speed, it's likely deployability.
I love Go. It's good for teams, deploys well, is enjoyable to program in, does things quickly. But if a client comes to me wanting a web application with forms, view logic, payments, 3rd party integrations – I'll take my Rails, thanks.
> Given all this, frameworks (and ecosystems) like Flask, Sinatra, Django and Rails still have an advantage for when you want to get something done today: they’re very useful toolboxes, handle a lot of the boilerplate and have a lot of examples and packages to leverage.
The author clearly recognizes this opinion – but there seems to be an idea that you can just "use Go" in place of existing web frameworks. That's not the case (yet!). I think this is largely driven by the myth of performance requirement.
[1]: Things like Node.js probably fit into this argument too
Nice arguments for "deployability" not something I initially would have thought to measure, since it's usually something that's "felt" later in the development/release process.
I guess it'll be interesting to watch how the frameworks mature and start to approach the Rails/Django or other micro-frameworks...
> The difference between the Go built in web framework at 190,687[2] (1.3 ms) json responses per second and Flask, a popular Python framework at 18,945[2] (3.1 ms) is trivial when you look at the bigger picture.
I disagree, and think that's a huge difference. The latency may not be, but requests per second? That could be a significant amount of difference if you start pulling decent traffic: you need ten Flask servers for every one Go server. That's a lot of ops complexity you can save yourself right there.
Flask, mind you, has a much more mature ecosystem (WTForms, SQLAlchemy, et. al) and development is likely to be faster on that front, but if you're looking at building something scalable and don't mind a bit of groundwork now, there is an advantage to using Go. It just may not be an advantage for every team (and that's fine: horses for courses).
> That's a lot of ops complexity you can save yourself right there.
If you're doing that kind of sustained request volume then you'd like have web server redundancy for availability reasons , right? Reverse proxying from nginx to a Go app vs. a flask/uwsgi app becomes a pretty small problem.
I'm definitely not arguing that Go isn't more "scalable". By all of our measures, it is. I'm talking about the huge sacrifices you make to productivity to use it as a developer for specifically modern web development.
I'd disagree that node.js fit that argument. I do python Django in my day job and a ton of node.js in my other projects (startup nearing launch, two other projects) and with express there is not a huge gulf in productivity between the two. All of my node.js projects is based upon No-SQL (leveldb, mongodb, couchdb) db's so I'm not up on JS's ORM libs but it might be a large difference there but it's certainly not when using a No-SQL db.
Gotcha. Yea, I put that in there because I've read/heard second hand information about Node.js requiring a substantial amount of boilerplate, etc. Sounds like that time has past. :)
I'm curious as to why everyone wants a web-framework in Go. Wasn't it primarily designed for back end service-type-of-stuff? Seems like there's plenty of full-stack and micro-web frameworks out there in what ever language you desire, that are in a more web-dev friendly syntax.
I guess I don't picture a company saying, we're going to use GO through-and-through for our dev platform. I picture it more as a "well, we now have a custom messaging service that needs to scale better" or something, much like when people choose to drop into C or C++. It also seems that there's more dev-friendly languages to build the web layer in.... I also don't see many web-dev types thinking in GO very well.
So, why GO at the web-layer?
For me:
• I prefer to find my errors at compile time. Consider the time to find and fix a missing method in a go compile (one 'make' and all the info is still in the developer's brain cache) versus eventually having a user drive the website into that method and having a mystery occur on a duck typed language. (hope you have good stack trace logging on the back end)
• I prefer a language that doesn't repeatedly break all my code. When I write and deploy a site, it should stay written. (Looking at you PHP. Granted, this was over a period of 15+ years, but all the more reason I don't want to go back and look at that code.)
• Efficiency is nice if you are ever going to hit a performance bottleneck. To elaborate: you get to defer the "OMG I need to map this load over more than one machine!" crisis for a while, and if you end up in a "I need N machines!", then making N smaller is good for your well being.
• Builds are fast enough that I don't care. I generally have a makefile for a website anyway to get everything organized and encoded, one more rule to build the go is no burden.
The weakest spot I'm finding is the html/template end. My strong typing evaporates at that boundary. I'd rather have the template get precompiled into go code and then compiled into functions with nicely defined interfaces on its parameters; faster to run and catch all those typos in field names. The current go mindset of a "go only", non-extendable, build command forecloses experimentation in this direction. (Unless you are a barbarian from the olden days, like me, and wrap everything in a Makefile.)
Cool. All valid points. I have a question, do you think builds in Go for a large web app will remain fast? I ask from the experience of working in Java with several minute build/deploy times...
From my experience, defintely yes. I use the Revel framework for creating my APIs to be consumed client side (Angular).
When I make changes, refreshing the browser results in a re-compile which takes less than a second (assuming I have modified several files) on my 2012 Macbook pro.
With Go, unit testing is also blazing fast. My entire test suite (around 30+ tests) takes around 0.2 seconds.
Just for comparision, I used Play 2 (Scala) for a short while before that. Re-compilation would take in the region of 10 seconds. Startup costs are high in the Java/.NET world :(
Thanks for the information. It sounds like you've got some experience that I can relate to. I might actually check GO out for some API work.
I assume this will be less of a problem in Go, because rapid compilation for large projects with complicated dependencies is one of the main design goals of Go.
Many other tools are good for web dev too, but having played around with it I'd say in principle Go is well suited to it, if you don't mind the lack of mature frameworks/libraries. I don't see why the syntax gets in the way of it being web friendly, which bit did you think would be a problem?
Deployment is far simpler with no libraries/gems/packages - single binary
Memory usage is an order of magnitude smaller than languages like Ruby or Java.
It's pretty fast (much faster than Ruby for example)
String manipulation isn't too painful - all the tools are there for manipulating unicode text and producing html.
Why not?
The "deployment" theme is pretty common that I keep hearing. Definitely something that sounds appealing.
Web servers aren't back end service-type-of-stuff?
I don't classify them the same. It's easier to have many web developers (in fact most I've worked with) who work on web app layer (working with the templates, javascript, depending on complexity of app the data-access-layer) than it is to have them working with messaging services, more complex data access logic, complex document rendering services, databased, our emailing management system, etc.
So, yes, I look at web server of an application as a pretty distinct part of the back-end, separated from many other complexities in the system. I also view it as the beginner (can't think of a better term [1] ) part of the back-end-system where there's a lot of devs who can conceptualize this layer who can't the other stuff. It's these devs whether I question they'll conceptualize GO in general.
[1] EDIT: I just thought of a better way to describe "beginner" with more clarity: non-computer-science-types who are competent at working in the web layer.
Because Go is a general purpose programming language and why not web-layer? There is even a tutorial on creating wiki site on golang.org.
Because it's fast, a pleasure to code in, and simple to distribute.
The distribution story is definitely appealing. I think it was only the other day a post showed up here with someone making command-line tools with it. I find this appealing.
One thing I'd like to see in Go is a way to sanitise HTML based on a whitelist.
This is to accompany blackfriday (Markdown) and text/template (templating).
Markdown permits HTML, and this allows some scope for nasty stuff to get in, or for bugs that may exist in blackfriday to be exploited leading to HTML that could be the source of a XSS attack.
We're currently running our user generated content through this: https://github.com/microcosm-cc/cleanse and more specifically this: https://github.com/microcosm-cc/cleanse/blob/master/src/main...
Which is a set of rules for OWASPs HTML sanitizer: https://code.google.com/p/owasp-java-html-sanitizer/
This works extremely well, except for the fact that it means in our Go code we're calling out to a process and asking for a Java process to be kicked up for each request.
If no-one beats us to it we'll be porting that to Go... but for us it remains a "When we need to" from a scaling or performance perspective.
Still... we'd love to see group work on a whitelist based sanitiser that we can contribute to rather than us go and write our own at some indeterminate point in the future.
In moving a site from PHP to go in a rewrite, I'm about to head down that same road. The plan is to use the HTML parser from the "not quite in the real distribution but kind of official" net repository:
http://godoc.org/code.google.com/p/go.net/html
Parse the HTML, walk the result, write that which is acceptable.
I have to restrict by tag, url scheme, and url server name in various contexts.
I looked at that myself, and decided that it wasn't the path I should go down.
ParseFragment throws an error on bad input, but actually I just want that stripped and to carry on processing things. If a user has put in a mostly usable piece of HTML and then got something wrong as an error rather than bad intent then permissiveness in how we handle that should rule.
And then I wondered about the wisdom of creating a potentially large security library on a not quite nailed down API.
Ultimately, given that this is a security thing, I figured it's best to go with the proven many-eyeballed solution that was had widespread acceptance.
Feel free to use the package we've provided, the bit of go code you need for it is:
import ( "os/exec" ) func SanitiseHTML(html string) (string, error) { cleanse := exec.Command("java", "-jar", "/usr/sbin/cleanse.jar", "--permissive") writer, err := cleanse.StdinPipe() if err != nil { return "", err } _, err = writer.Write([]byte(html)) if err != nil { return "", err } err = writer.Close() if err != nil { return "", err } buff, err := cleanse.Output() if err != nil { return "", err } cleanse.Start() return string(buff), nil }Thanks to a fellow HN reader, we now have a head start on trying to create a HTML sanitizer.
https://github.com/microcosm-cc/bluemonday
Feel free to use it and help out.
Initial work was all done by Matt Jibson as part of his Google Reader Clone: https://github.com/mjibson/goread
From someone who's betting the farm on writing a web application in Go (https://www.harrow.io/).
We've chosen Go for two really significant reasons: a) to reduce attack surface area and b) to enable us to distribute binaries to customers who want an enterprise/firewall install inside their own network.
Speaking about surface area, we've reduced the amount of framework code in stack (rails, gems, etc are all off the table) to zero, and we've reduced the app paths to a few tens of lines of code, in most cases.
Whenever one trades off a framework "to make things more secure" one has to be sceptical, there's a massive scope to introduce SQL injection, XSS and other nasties. In Go, in our case this is fairly limited. So take that reason with a pinch of salt if you aren't building a VERY well defined app with VERY small and skilled team. I don't expect this to scale well.
The second part is also a niche value, whilst it will have some benefits for us running our hosted version of Harrow, it'll have more profound benefits when we can sell binary installations to customers.
Finally, learning how to build applications in an API centric manner, without the crutch that is Rails has been interesting, and has made us all better programmers, we have a more interesting architecture now, and are making better use of our tools (we're leaning heavily on PostgreSQL, triggers, stored procs, LISTEN/NOTIFY etc) and we have a faster system.
On the flip side of things, timing attacks are now really something we have to be mindful of. For instance the login endpoint returns "404 Not Found" regardless of whether it's an incorrect password, or the account doesn't exist. Except, interestingly we're using bcrypt (we're switching to pdbfk2 before the public launch) we need to look up the user before we can hash and verify the password. Meaning we respond in < 1ms when the user is not found and 4ms when the user is found, but the password is incorrect. (and 4ms when the password is valid). This also applies to a lesser extent in responding with "404 Not Found" when trying to snoop on someone else's project/logs/build/etc. The same way they Github does. But it's less critical there as there isn't the artificially long password hash comparison step.
All told the decision to build in Go has made the development of the project slower, but has in my mind at least lead to a better quality of software, with things that don't typically show up in Rails/et al projects (foreign key constraints, triggers in the DB).
Of course all these tools are sharp weapons if handled incorrectly, and testing triggers/functions in the database can be a nightmare from a unit/functional testing point of view.
We've avoided technology proliferation (the entire stack is Go, and PostgreSQL, with worker nodes which rely on LXC). The entire application uses three packages not taken from the Go standard library. (db driver and bcrypt library)
Ohh, and our test suite compiles, and runs (all of it.) in less than three seconds. That hasn't hurt much, either.
Just an FYI, noticed two typos on your homepage: "easyily" => "Easily", and "ause" => "use".
Interesting approach to the stack, I'm considering the same for some side projects. I like the exercise in a fully API driven approach. A major advantage is the pure separation of client and server means refactoring/changing languages becomes more straightforward you can also isolate data/logic from presentation more cleanly.
Great catch, thanks - we're not pushing the homepage too hard yet :) so my pride isn't too badly hurt. Rolling a replacement soon!
Anyone have a link to an open source web app of a non-trivial nature built in go?
I've seen lots of system utility stuff, but I don't think I've ever actually seen anything other than how-to-use-this-framework web apps?
There is https://github.com/peterskeide/bones (I'm the author). It's by no means a large codebase, and it's not finished, but the goal is to give you a decent starting point with some hopefully useful idioms to build on.
I think Go is still missing its killer-web-app-framework. But maybe it's better that way. Keeps out the people who don't like think before they code.
Most people seem to define killer-web-app-framework to include a "blessed" ORM solution. (I don't personally, but I understand where people are coming from.) Go seems to have been resistant to any large-scale ORMs to date; it's hard to put why into a quick HN post, but the language seems to sort of resist it, despite having some reflection capabilities.
Or at least, the language resists the sort of ORMs that the web world has become accustomed to, with tons of convention-over-configuration and such. A more static one could probably be done.
I find myself wishing the "go" toolchain would give me a defined way of running certain code before the compiler, so I could generate Go code with the go source tools. From there you could easily compile yourself up some ORM code by examining a database at runtime. It's yet another privilege it claims for itself but gives you no way to hook into.
I've become a real fan of Flask and the core framework does very little beyond managing request routing and having a simple framework for using jinja templates. Everything else should be optional. My day job is Spring MVC which is an absolute behemoth. It's an application framework more than just a web framework and the plethora of gadgets are useful for alleviating a lot of architecture decisions, but I'm sure I could get by without them.
This article seems to be pushing Martini, which at first glance looks interesting (though it doesn't appear to have been mentioned on the mailing list even once, so there may be some boosting involved).
There is Revel, but it seems to be getting a bad rap ("non-idiomatic" label used as a stick).
Martini author here. Weird how it didn't show up in the golang-nuts search, here is a link to the announcement topic: https://groups.google.com/forum/#!topic/golang-nuts/LubMauy6...
Martini is still young (less than a week) but the core is pretty fleshed out.
No boosting here (I don't even have that many twitter followers). I just put together a framework that doesn't step on your toes and people ate it up :)
I love what I've seen so far. The biggest problem is probably the age. Googling for "go martini json" will not yield results until more users start to talk about it.
Yup. Getting there will take a while. In the meantime I plan on releasing a whole bunch of middleware in the martini-contrib repo and creating more video tutorials on web development in Go.
I mentioned Martini as I like its approach. It hit HN a few days ago and was on the go-nuts ML around the same time.
I don't expect everyone to agree with me surrounding Revel, but I certainly think it's lack of modularity is a negative.
I had searched for it to see what kind of feedback it was getting but came up empty:
https://groups.google.com/forum/#!searchin/golang-nuts/marti...
The Revel comment was more about a criticism I've read several times now.
I liked and bookmarked the article. :)
When I read all these posts praising Go, I feel like a stranger accidently appeared at someone's birthday party. I really want to start loving Go, it looks perspective and people already benefit from using it, but I just cannot make me enjoy the language.
Having Java/Scala background I used to have static checking and performance for free and I cannot feel about Go the same as someone who switched from Python, PHP, Ruby stacks. I code mainly in Scala, and even if I like Go's simplicity, compilation speed and the absense of Java burden, I really miss features like default immutability, pattern matching, generics, reach collection toolset, powerful typesystem. The language looks just not expressive for me to really enjoy.
Maybe I am corrupted with FP, cannot value simplicity or I should focus more on getting things done not on the process of writing code. Just wonder, if anyone else have similar impressions / feelings?
All this talk about HTML templating (html/template, etc) & Go web applications, is missing any discussion of what IMHO is a better way to write Go web apps:
1. Write a backend webservice in Go's net/http + Gorilla
2. Use something like angular JS to interact with those webservices. (From the server's perspective this means static HTML/CSS/JS assets that can again be served by Go, from memory if desired).
My first couple Go web applications did things the html/template way and were structured on the way I used to go JSF applications- but since I switched to the above approach my projects have been better performing, faster to develop , better scaling and easier to maintain. As we all know, we seldom get all those things in one technology choice without trade-offs.
That's the approach I'll be taking when writing a future side project. Fully API driven with go. Angular/Grunt/Bower for frontend, perhaps served by go or possibly on a separate machine serving just HTML, just to see how it performs.
I've even considered Github pages for the HTML. But probably not for production :)
Author of Martini here. I think right now is a very exciting time for web development in Go.
One of the goals of Martini is to rally developers to create some more reusable web modules in the form of handlers and middleware. Martini has gotten a great response because it doesn't infringe it's api on your modules/middlewares.
I personally think that this is the way to move forward with web development in Go. Less coupling, pull in the dependencies you need, get stuff done.
I've came to like some of the components of ghost: https://github.com/PuerkitoBio/ghost
Also worth mentioning is Hugo, a static site generator: http://spf13.com/project/hugo
Not in the post: the go-start framework https://github.com/ungerik/go-start
Not in the post: beego https://github.com/astaxie/beego
I guess there's a good reason for that...
Yowsa. No thank you.view := Views{ DIV("myclass", H1("Example HTML structure"),
What about web.go? I never tried it but it seemed promising.
As touched on in another HN thread: web.go isn't "actively" maintained by the author. Whilst any of the packages mentioned in the article could succumb to the same fate, I made an effort to highlight some of the longer running projects. Gorilla is probably the cornerstone here.