Why We switched from Python To Go
medium.com> #2 Static Type System
It's really weird to see one of the reasons for their switch to Go was because it's a statically typed language. If you want static typing, you don't choose Python in the first place. You know it beforehand and it shouldn't come as a surprise for you.
> For example it has http, json, html templating built in language natively
So does Python with urllib and json modules. I don't know if it has HTML templating in the standard library, but you can always use jinja2.
> Great IDE support and debugging
Not even a mention for the excellent PyCharm or so many of the useful Python vim plugins?
>If you want static typing, you don't choose Python in the first place. You know it beforehand
Well, people begin using languages for different reasons and it's possible that there was never a conscious explicit choice of "dynamic vs static". Instead, they were actually interested in some specific task library (content management system, numeric computing, scrape webpages) and the underlying language happened to be Python.
For example, a programmer might stumble into Python because he was using Django, or work colleagues used Numpy/Scipy, or the easiest HTML parser was Scrapy. That the language happened to be Python and it happened to be "dynamic" is incidental. In Django's case, that project is 4 years older than Golang. Maybe Java Spring was too verbose but the new Golang with static typing (and less-verbose type inference) hits the sweet spot.
Probably the most widespread example of "dynamic vs static" not being a choice is Javascript. Millions of us didn't program Javascript because it's dynamic; we used it because it's the only language available on all the browsers.
Also, some programmers may not realize they want static typing until they experience pain points with dynamic languages on large projects. (E.g. easier refactoring.)
> It's really weird to see one of the reasons for their switch to Go was because it's a statically typed language. If you want static typing, you don't choose Python in the first place. You know it beforehand and it shouldn't come as a surprise for you.
It's weirder that many Python/NodeJS switchers cite this reason, since out of all the commonly used statically typed languages, Go has the second weakest type system (the first place obviously Goes to C).
With so many methods returning an interface{}, you'll get exactly the same crashes you had with Python.
If type safety is really such a major issue for you, you'll get a significantly better deal from Python Type Hints (using mypy or PyCharm) and TypeScript/Flow. All of them support generics (and hence nearly eliminates the need for wildcard types).
I suspect performance and static binaries are the real reason for most of the cases of projects switching from dynamic languages to Go.
What functions return interface{}? I rarely see it outside of marshaling/unmarshaling (where I believe it would remain even if the language had real generics) and contexts (which would also use interface{} if there were generics).
Sometimes you want dynamic typing, how else will you pretty print structs? The compiler could generate code but that's a lot of bloat. For context, the point of it is that you don't know what's in it, it's an opaque container for information to be used by middleware. The fact that data might be missing lends itself to easier refactoring later on.
I want generics in go, but I don't find my code using much interface{} at all.
It really depends on the type of application you're building, I suppose. I've seen applications littered with empty interfaces, and others with hardly any.
If you really want good static typing while being fairly python like, there are other great options. Java or c++ both have decent generics, speed comparable to go, great libraries and years of collective experience using them.
I truly hate the social hacks that are causing go to displace much better languages.
It's the learning curve. If you know another imperative language, you can grasp golang in a week. With C++ or Java, you need months.
Java is pretty darned graspable, at least I think it's way closer to Go than C++. What do you think are the biggest things that make it hard? The language? Warts like int/Integer? Library size or organization? (This is a real question, I can't look at this stuff with "fresh eyes.")
For me? The environment/runtime.
I can write a simple program in C/C++/go/rust/etc, and compile and run it.
In Java, a bloated IDE isn't just a recommendation, it's practically a requirement.
Evey time I try to learn Java, I smack my head against the toolchain. I ended up just giving up when I tried to wrap my head around classpath, and haven't tried since. I could probably figure it out pretty easily now, but I just don't see any allure in learning Java anyway.
One of these days, I might start using Clojure, but the JVM just feels so messy that I don't want to start.
There is no requirement to use an ide in Java. I've never written Java in anything other than emacs and never had a problem.
I'm likely overstating the problem, but that is my experience as I remember it.
My main point is that Java just isn't appealing enough in the first place for me to want to deal with the relatively insignificant issues I ran into.
> much better languages.
I disagree. Java has a bloated syntax, and C++ is a mess. Java also depends on a proprietary runtime, with mostly compatible free alternatives.
The biggest advantage C++ offers is speed, through memory management, which is usually a pain to deal with.
Go, on the other hand, has a well thought out syntax, good support for concurrency, garbage collection, and compiles to native executables.
Could you illustrate what you mean by "bloated syntax"?
Everything else you mention (besides native executables) Java also has. From what I can see, Java provides far better concurrency support - more paradigms than simply go routines.
Classes are required. Sometimes I just want a c-style program without a class. Java looks a lot like C++ (just with garbage collection, and some more consistently defined behavior). In comparison, golang (and rust) have some nice syntactical differences: inferred types, postfix type annotations, etc.
Go routines are nice when that is all the complexity you want. Having more paradigms isn't an interesting prospect until it becomes specifically useful.
Java may be more usable in many cases, but it, in my experience, just isn't easy to work with.
Apart from the boilerplate enclosing class (4 lines, vs 2 in C), you can write a C-style program without classes in Java. There is literally nothing stopping you and generics are helping you significantly for this.
That's completely true, but I just don't like the boiler plate. It's not hugely important to the grand scheme of things, but the more barriers I have from writing a function to running code, the less I want to deal with the toolchain/language in the first place.
About the static type system:
When people say they like Go's static type system sometimes they are comparing it to something more verbose like Java.
Think about someone that knows Python and have to use Java, then they think that all static type languages are as a pain in the ass. So they begin disliking static type systems when they really just hate the Java's one.
And so the moment they are using Python and start playing with Go they find a static type system they kinda like because it's less verbose and without so much ceremony.
> When people say they like Go's static type system sometimes they are comparing it to something more verbose like Java.
Go's system throws out a lot of the ceremony of older Java, but also throws out a lot of the benefit; many modern languages (including recent Java) also use type inference and other features to reduce static typing ceremony, but retain at least the power of the classic Java type system, and often far more.
Go's type system seems to be a poor benefit-to-ceremony trade-off.
Java the language is verbose, sure... but the type system?
Well, having a static type system is a tradeoff...
When using Python I feel the least resistance from the language, you can do a lot with a little code. Problem is, when the system grows you begin to miss the checking that static typing provides (esp. when doing refactors).
When coding in Go I feel that there is a little productivity lost upfront, but when your script begins growing and starts to look like a "serious" project you feel more confident changing your old code. This is because some kind of errors are caught by the type system before doing manual/automated testing.
And I'm talking about personal projects where I'm alone, this could be more important in projects with several coders.
I think yupyup is referring to the fact that all types must be explicitly annotated.
(I favor this and explicitly annotate most of my types in Haskell and Scala. Makes reading easier. But I did occasionally find it annoying in Java when it was mandatory.)
Inferred types are nice.
They make cleaner, and therefore, more readable code. That is the main thing people want from dynamic typing anyway. It's really not very common to change the type of a variable, and a statically typed codebase is more predictable, and therefore easier to manage.
> It's really weird to see one of the reasons for their switch to Go was because it's a statically typed language. If you want static typing, you don't choose Python in the first place. You know it beforehand and it shouldn't come as a surprise for you.
Why is it weird? You have to have experience with statically and dynamically typed languages before you can choose the one that will fit your project the best. If you don't have the experience, you probably choose the one you are familiar with. In a world where everybody knew everything, everyone could make the right decisions and none of the wrong decisions. That is not the case, though.
"If you want static typing, you don't choose Python in the first place. You know it beforehand and it shouldn't come as a surprise for you."
People change. Dynamic typing was popular some years ago, but opinions have changed.
Projects change too. What's good for a small project might not be for a big one.
This is weird. It's not a matter of what was popular a few years ago.
Static typing didn't "stop" being popular while dynamic typing "was" popular, nor have opinions "changed". Dynamic and static typing have coexisted almost the entire history of programming and will continue to do so.
And so will the flamewars about them, unfortunately.
"Static typing didn't "stop" being popular while dynamic typing "was" popular, nor have opinions "changed". Dynamic and static typing have coexisted almost the entire history of programming and will continue to do so."
That's a non sequitur.
How so? It was directly addressing this:
> Dynamic typing was popular some years ago, but opinions have changed.
> Not even a mention for the excellent PyCharm or so many of the useful Python vim plugins?
I must also mention Python Tools for Visual Studio is REALLY good. I have used PyCharm and PTVS and I must say PTVS is absolutely fantastic. PyCharm wins for me in the Cross-Platform aspect though.
I haven't used Jetbrains' go ide, but extrapolating from my time with intellij and pycharm it seems that their ides work significantly better with static type systems. (Find usages, rename function, etc are less accurate.)
PyCharm does support type hinting now which should help: https://blog.jetbrains.com/pycharm/2015/11/python-3-5-type-h...
> It's really weird to see one of the reasons for their switch to Go was because it's a statically typed language. If you want static typing, you don't choose Python in the first place.
Sure, if your only or most important criteria is static typing. But you can view static typing as desirable, but prefer Python because it has more important positive traits that no alternative with static typing has, so you sacrifice static typing for them. But then Go comes along, that is good enough on those criteria and has static typing, and you prefer it.
Of course, Python now has optional static typechecking available anyhow.
I've tried moving one of my projects from Django to Go before. I must accept that I had to wire a lot of stuff in Go, that Django handled for me like a cake. I'm talking about configuration, patterns and common features.
Others who've been where I was: how did you cope up with this? Would you be ready to move another project from Django to Go, without the mental fatigue?
For the majority of my career I haven't worked on greenfield projects. I've been debugging, modifying, and otherwise maintaining pre-existing projects. As a result I've grown to really dislike frameworks that autowire things together. The very same features that made it easy to get started make them harder to maintain going forward. It turns out following the code in your own codebase is easier than following the code in the framework.
Django isn't as bad as some in this respect, (cough, spring, cough), but it still exhibits some of the same issues regarding maintainability. Go front-loads some work so that longer term maintenance is easier. And the vast majority of code work is maintenance not greenfield development.
Just my personal anecdote, but I recently had to port the backend of one of my pet projects away from ruby due to performance issues - parsing around 50MB of string in ruby isn't an amazing idea.
So I figured the natural choices there would be go because it's new and in demand, and java because I have a couple years of java se experience.
So, figured, let's evaluate java... and I just failed to comprehend the spring stack just from googled tutorials. There's such a dense layer of magic going on in there, I'd either need to not understand most of my stack, or spend weeks and weeks reading and understanding.
Yeah, Spring is one of the worst offenders I've seen. At my current job I'm working on code written with Spring and debugging takes probably 5 times as long as it should.
Reading your comment reminded me of this great talk by Christin Gorman about maintainable code:
(Jump to 25:50 for an anecdote about maintaining a "well engineered" system)
Oh my goodness. She totally captured my career in a nutshell. There is nothing quite like running into an interface while you are trying to trace a problem. Your head just immediately hits the desk and then you go looking for aspirin.
Point 3
Goroutines are more performant than python threads if you don't understand the difference between a thread and a routine.
How about asyncio in python 3 and gevent etc. in python 2?
Right I'm not saying that python 2/3 doesn't have these abstractions (to be honest I'm not an expert in either Python or Go, I'm a C++/Haskell guy). Just that the author is creating an obvious false dichotomy.
Callbacks (asyncio & gevent) are not the same as coroutines. You have to experience the callback hell to understand the difference.
asyncio and gevent are coroutines, at least in all the senses that goroutines are.
In fact, so are regular post-PEP342 generators.
Asyncio does not use callbacks. It uses async/await syntax
Got you something faster than go, in python... switch back to python :P
https://magic.io/blog/uvloop-blazing-fast-python-networking/
So this lists the usual stuff that people who aren't experts in any system think are good reasons to pick one or the other for.
1. The single binary is attractive, but with modern virtualenv and wheel and anylinux wheel files this is far far less important than it would have been 10 years ago. I count this as irrelevant but probably python's library packaging is confusing to a beginner.
2. Static types don't really help you here, yes you can run into some issues where you have an object that is a different type than what you thought it was, but this is not common and your tests will catch it for you. I'm not sure why django's orm will fail on the wrong type since SQL is loosely typed anyway, but everything django is garbage so don't use django.
3. Performance in python can be pretty awful. Not going to argue. It's usually not too slow and you should focus on algorithms first, if you agorithm is efficient then you should be able to do whatever computation you need to render html or json for your service in plenty of time. But Python is slow sadly.
4. You don't need a web framework for Python either, Werkzeug is a great toolkit to roll your own micro framework. So I'll just count this as lack of experience and knowledge, plus confusion.
5. Great IDE support. I guess this is a thing people like to have, personally I think if you are being slowed down by how fast you can type or remember method names something is horribly wrong. The other things IDE's do are usually pretty pointless, at least for me, but if you feel like you need it then I agree IDE support for python is usually pretty weak because python is very hard to statically analyze. None of the more productive programmers I've known through the years used IDE's, they all used Vim or Emacs. If you feel like you need hand holding I guess IDE support would be important for you though.
1. There are other ways to do it, yes. But "just upload anywhere and it works" is not just for server, but for tooling. That what comes out of the compiler can be used everywhere is a huge selling point that also, unfortunately, helped PHP even though there were other ways to do that too.
2. I could not disagree more. Just costs between int, float, int64, and double cause lots of bugs. Go makes costs explicit, which may be annoying, but does fix issues. Also lots of python code needs to check if passed args is str, or Unicode, or byte, or list, or dict, (or does, anyway, even though it shouldn't) so when someone passes in a generator when code assumed, or checked for a list, then all hell breaks loose.
4. I disagree with the article and think that the gorilla Go router is so good that you pretty much need it. :-)
autocorrect. "costs" should be "casts"
As with all "why we changed from technology X to technology Y", the story is that there was this or that perceived shortcoming in technology X and technology Y will be our savior and fix all our problems, or conform to our software ideology/dogma.
I used to read these things but given that I've read one, I've read em all.
The only reason people keep writing these things is blog filler to promote their company or do indirect recruiting. Surely cannot be cause anyone cares why you use a blue pen instead of a black pen.
> Surely cannot be cause anyone cares why you use a blue pen instead of a black pen.
Actually... I think folks do care. But I also think that depends a bit on how you look at what's interesting. Some people want to use new languages or different languages for the sake of them being new or different. Others are more restrained but will reach out to a new language or experiment with it if they have some reason to assume this language can solve their problem better (for a definition of better). Others are set in stone and won't consider moving on from what they know.
What makes some of these posts interesting is when it goes much deeper and has thorough details that show how switching to a different language improved the situation for them, on the axis they were looking to improve. With actual metrics, crashes in production for example, deployment size, time from commit to deploy, resource usage etc. etc.
This article of full of fluff. Citing a static type system as a point for Go? Go's type system is about the weakest of any popular, statically typed language being written today. It's full of escape hatches.
> It's full of escape hatches.
For people who are accustomed to Python or Javascript this is a feature, not a bug.
It's neither, it's a bad design decision that they were forced to make because the type system isn't expressive enough.
As an aside, the greatest thing about Python is StackOverflow one-liner solutions to common problems. How do you find your developer productivity fares in Go? To take a simple example, I was experimenting with C++ and had the simple task of "reading in a CSV". This simple C++ task does not have a nicely formatted, pre-approved, community rubber-stamped, best practice 500 green ticks StackOverflow top accepted answer (sometimes by Core Python / prominent PyCon developers) that I can just copy and paste.
> How do you find your developer productivity fares in Go? To take a simple example, I was experimenting with C++ and had the simple task of "reading in a CSV".
Not sure if reading data from a csv formatted file is a great example for differentiating Python and Go: both languages have a builtin csv package/module.
Python: https://docs.python.org/3/library/csv.html#examples
Golang: https://golang.org/pkg/encoding/csv/#example_Reader
And, errors handling put aside, both takes more or less the same number of LOC.
> And, errors handling put aside, both takes more or less the same number of LOC.
So apart from the difference in line count they are the same line count :)
Right, it's almost impossible to get any work done without having pre-approved code to copy and paste from. This is especially true for highly complex problems like reading a CSV file. Those virtually require the presence of promiment PyCon developers as no one else can possibly get this right.
Only use code with at least 500 green ticks.
I always appreciate a healthy dose of sarcasm, but I think you're missing the wider point about the great community we have in Python - not only in terms of the ecosystem, but also continuous guidance on best practice for common problems.
"How we rationalized after the fact that we wanted a new toy"
There are many good reasons to switch from Python to Go, but that article misses them all.
What are those reasons in your opinion?
You identified that your code has few data parsing and manipulation, and a lot of concurrency bottlenecks after auditing your system and making measurements.
You have a lot of technical debts in your concurrency code and don't think you have the skills to solve them internally while Go would let you do it.
You have data about your deployment and weighted security updates easiness against self enclosed binaries and the later is more appealing.
You evaluated the cost of a rewrite in the current used language vs the new one, including training and documentation and it was in favor of Go.
You have embarrassingly parallel operations the market / your resources forces you to optimize and the serialization cost of data makes it impractical with multiprocessing.
You want a sweat spot between verbosity, high/low level, productivity and performances and don't mind having something average in every of those aspects as long as it's balanced.
You are aware of the cons of Go (very heavy error handling, skinny stdlib, youth of the ecosystem, etc) and it's still worth it.
You made a prototype of a basic task you had in the previous tech, with the last version of the previous tech and Go and the ratio "cost / result" is clearly in favor of Go.
You have experts in Go in your team and their talent would largely compensate the lost of the talent in previous tech.
You deeply checked Java, Erlang, C#, Haskell, Rust and JS as alternatives and your team agreed Go would still be better.
You checked solutions to your problem (e.g: for python, you tried crossbar before being sure you can't do your microservices properly) and damn, Go still solves them better.
You know how much RAM your service can afford, and spawning goroutines likes it's going out of style will be the proper ratio of easiness vs cost.
Basically all this posts says is "x is good so it's cool we have it". Good reasons to switch are never black and white, they are justified by experience and data, in a specific context, relatively to pros and cons of other solutions.
Anything else is just geeks being geeks and wanting cyber poneys. I understand that, I do it all the time. Loving your job and trying new stuff is a perfectly reasonable motive in my book
But I'm not going to write a blog post about it pretending I had technical reasons while I did clearly zero measurements and I am just quoting what other people say about Go.
The blog post sounds more like they had lunch and they said "dude, we should totally rewrite that in Go" and went for it. Then they measure a newly created service against legacy code and say it's better. Well, guess what, if you make a successful rewrite of anything with the same people, it will be better because you had the experience of the previous one.
The clue here is when they said they had less code in Go than in Python. This is clearly a red flag.
Plus if you had only 35% gain of perf from your Go system, something is wrong. With pypy only I can gain up to 800% of speed on my 2.7 system without changing a line of code. In 3.6 with libuv and asyncio you can also get a huge gain on IO operations.
Done properly, you should have way more gains from a Go rewrite. Or your problem didn't need Go.
No research has been done here.
Seriously thank you for this. So many "Python-Go" articles are indiscernible from Google marketingspeak and just sit on the front of HN without being called out. Your list of tests will be a great reference.
The lens through which I resolve Go is "it solves Google's problems". These relate to a systems programming codebase in Python and C++. Go makes clear sense for parallel contexts (though nobody has Google's problems). In other use cases, it is more or less just another general purpose programming language and the engineering tradeoffs are less obvious and the programmer ergonomics are more a matter of taste.
That's very true. In fact, for most new projects you will never reach the problems or scale to benefit from Go sweet spot. JS will give you the "one language to rule it all the web stack" benefit. Python will give you the "good for almost any task" benefit. Rust will give you the "safe bare metal language" benefit. Erlang the "unkillable service" benefit. Java the "huge pool of talent" benefit. Etc.
For Go, it's not as easy to justify it so you should spend some serious thinking time before investing on it. The reward can be great, but the cost just as well.
Excellent list. Should give everyone a good start. I wish we had more of these for the different languages, that would at least make some discussion around these "switching from X to Y" posts more fruitful.
Not the original parent, but my reasons for switching a couple of projects from Python to Go were primarily performance and ease of deployment (E.G., building a statically linked ELF takes a few seconds).
The syntax and ecosystem weren't really critical factors, but writing concurrent code is a lot more fun in Go.
Keeping your engineering team happy by allowing them to play with a new toy.
If someone switches away from Python he never used the unprecedented expressiveness of it, which is tens of orders of magnitude higher than the next best language.
One line in Python is worth several screens in the next best language.
Python can be as fast as C++ if you use all its features.
Development time is around 20 to 50 times faster than Java.
That's a dumb article. Take point 5 for instance: You can code in Go with Jetbrain's IDE. Well guess what, Python has a dedicated IDE made by JetBrain: PyCharm. How is this an advantage ?
A copious amount of misinformation in this article. Another honest? effort from the stable of individuals with a shallow understanding of elementary CS.
> Python is great and fun language but sometimes you are just getting unusual exceptions because trying to use variable as an integer but it turning out that it’s a string.
Oh why would I get an unusual exception, if I tried doing that _without even a try except block_ hust, I wonder ... must be because of the language itself and how it works.
> [static typing]
Why would you have started with Python, if you did not like dynamic typing? Also I doubt that static typing saves any time at all. It might also depend on what you are used to.
> [performance]
I am not sure what the application is for what Go is used in this case, so one cannot really talk about the necessity of speed in the authors use case.
However, the static typing example was a Django thing, so might be it is about some web app. Web apps are typically not the most performance needy things around.
Before bringing up the performance argument, one should check whether the performance of the language is the bottleneck or rather some I/O, like reading from disk, database or network (for which the appropriate libraries are responsible, like a database driver, which is likely to have similar performance across libraries).
If it is really about some calculation being done in the language itself, there are many high performance libraries available for Python, which are implemented in Fortran and C, so I doubt that in a proper setting using Go instead of Python and its libraries would have much of a lead in terms of performance. If you are doing massive matrix operations in the language itself, then it is sort of your own fault. Python is a language, which lives from its rich ecosystem. If you don't use it and look out for stuff, which could help you, then yeah ...
> [built-in http json ...]
Many of those Python has as well and since when is built-in automatically better than an available library, if the source code of the library is properly checked, before going to the official repositories? From a minimalistic point of view, one could even argue, that it should not be part of the language and that one can "customize" ones setup depending on the task at hand. If looking at the time needed for setup, it is only once a creation of some virtual environment (virtualenv, anaconda, whatever else there is) and you are done.
What happens, when there is an update to Go or a series of updates, which add a new feature, for example like Python's `async` and you want that in your application? You'd need to update your language compiler. But what if some of the so nicely built-in libraries changed as well and is now incompatible with what you wrote before? Maybe some function had a bad name and got renamed. Bam! You cannot update your language compiler/interpreter without breaking things. And why is that? Because the built-in library is not decoupled from the language itself. If it was not built-in, you could have it tell you, that it is only compatible with some version of Go. That's how it works with Python's libraries in anaconda environments and it is all automatically taken care of for you.
> [IDE]
Meh, Python got those too. I am still using Emacs amd Vim instead of an IDE. It didn't make me any less productive.
Next up from The Author:
5 Reasons Why We Switched from MySQL to Postgres
Why We Switched from Postgres to Oracle
11 Reasons For Switching From Oracle To KDB
7 Reasons Why We Switched from Go to Fortran-77
Grammar errors are provided for free by The Author.
Thought the same. Plus: "Surprises" that should be obvious for everyone working with the discussed technologies...
I guess we'll have: "This will surprise everyone, but Postgres can actually handle GIS data better than MySQL".
Well Medium strikes again. Every post on this platform seems to be utter bs.
Lol!
From the HN guidelines:
If the original title begins with a number or number + gratuitous adjective, we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to "Ys."
His grammar is atrocious, yet this is one of the clearest and most persuasive things I've ever seen on medium.
I learned an important lesson about writing from this author.