There’s a myth that Perl is slow and memory-inefficient. What I’m currently doing involves manipulating some data in nested hashes, it’s relatively easy to benchmark, and I have some spare time. Let’s find out if there’s a better tool for this!
The Task
Generate a map of a specified depth, taking keys and node ‘width’ randomly, pretty-print the tree if the depth is less than 4, then count different keys in the tree and print the results. There are quite a lot of quirks in here, so please consult the solutions for details. (Yes, I’m lazy.)
Disclaimers
- I am not doing any crazy optimization in the code. In fact, I aim for clear code above anything else (otherwise I could’ve used in-line assembler, do some cool parallelism or replace hashes with arrays). The code should be what would you normally write without aiming for speed. Or, to put it another way, what would you write at a job interview.
- I am also not changing the basic algorithm of my test, although the results produced can be obtained much, much faster and with less memory consumption.
- The compilers or interpreters, on the other hand, can do whatever they please, and I am turning on their optimization options I can find. They can even pre-calculate the execution result if they figure it out (no one could yet, though).
- I am aware of the Programming Language Shootout. I don’t think they adhere to (1) and (2).
Here are the results, amuse yourself but please read on afterwards.
After writing several implementations I actually realized the benchmark is only a part of the story – and this is actually why I’m writing the post.
Perl and Python presented little to no difficulties, other than making sure they have the same amount of data to process – so I had to implement the same PRNG in both.
The next one was PHP. Of course, I just copied and amended the Perl one and run it. PHP stayed true to its philosophy of “a wrong result is better than no result”, spit out two screens of warnings and errors and some garbage instead of the result.
Haskell. No complaints, really. It died early with a stack overflow when asked to process a large amount of data, and happily ate all my memory several times after I increased the stack size – but it’s been a pleasure to deal with.
C++ greeted me with a segmentation fault – easy enough, the error wasn’t even in the main task, but in the command line processing. (But read on for C later…)
JavaScript (using Node.js) looked like a good idea, with all the hype. No problems with expressing myself, good speed… but PRNG implementation copied from C++ failed. Turns out all numbers are floating-point, and their precision is not enough for the bit crunching it does. Really? This is the new everything language? And what do I do, since I still want to compare it to others (although this very much turned me off it)? Well, I rewrote the generator in all the languages to be so stupid even JS can cope. After some parameters tuning, we’re back on track!
C. This wasn’t going to be easy – I only started this when I though of a plausible approximation of a hash map. Of course, it segfaulted a lot. Of course, it leaked memory, Of course, it took Valgrind to understand what’s going on. And I even got a Frankenbug – only the optimized (-O3) version of the program crashed, while the debugging one ran fine. Well, to cut it short, I forgot to return a value from a function. And by the way, while doing this I even found and fixed an an embarrassing mistake causing a leak in C++ version. Not that it changed much in the rating.
There was Ruby and Java I didn’t mention – but this was so smooth I’ll only say Ruby 1.9.3 is faster than 1.8.
Conclusion
- Python rocks the interpreters. (No, JavaScript, you’re fast but can’t even do math.)
- My C skills (or, rather, map implementation) didn’t beat STL – no reason not to ++. Or, actually, Java is as fast.
- Haskell eats up a lot of memory. I suppose profiling might show something here, but I’m not making an exception for it.
- The results seem to differ on 32- and 64-bit architectures.
Final Words
The benchmark can be done better. The solutions themselves can be done better. There are a lot of languages I would like to see here (.Net, LISP/Scheme, assembly if you are brave enough and my C result doesn’t discourage you). C and C++ solutions cheat: they store characters, not strings. Yes, I am taking pull requests (but see disclaimers).
The Source