Jubilee, a new Rack server for JRuby based on the high-performance Vert.x
github.comThis is awesome! I would love to see some benchmarks compared to say, Puma. I'm a huge fan of Vert.x and I love this!
Edit: The demo chat application looks totally awesome! http://192.241.201.68:8080/
warning: there is no script escaping so someone malicious could redirect you to other sites, etc. use with care.
Thanks Ryan :)
I am planning a big ruby benchmarking blog post in the next week or so. I will definitely add Jubilee to the existing list of rack servers to test with!!!
Right now I plan on testing rails, sinatra, cuba, plain rack, probably the frameworks previously benchmarked here: https://github.com/luislavena/bench-micro
I plan on benchmarking on thin, unicorn, puma, WEBrick, torquebox 3, torqbox (torquebox 4), trinidad, passenger, and I guess now Jubilee.
I was also thinking of doing some EventMachine based stuff like goliath and whatever else I can find in that space.
From my early benchmarks last weekend torqbox is the fastest and pure rack is a lot faster than sinatra or rails. Sinatra is not nearly as fast as I thought it was compared to pure rack. A simple thin server running ruby 1.9 got about 1500 req/sec for hello world using Sinatra. Torqbox and jruby brought that to about 5000 req/sec. Plain rack was about 6500 req/sec on ruby 1.9 and thin and on torqbox it was 10,250 req/sec. For reference, my golang hello world was getting 10,500 req/sec.
I'm very curious where Jubilee will land in that mix.
One more ruby framework that might be interesting to test would be grape. https://github.com/intridea/grape Thanks for the work you've done with bench-micro. I was not aware of all of those frameworks.
I can't take credit for bench-micro, but I was also really excited when I found it. And yes I'll add grape to the list.
I'd be curious as to where Reel (both on and off Rack) lands in there as well.
Be sure to check out Reel:
Oh fun, I'll have to add that to the list.
Would be really cool to see it tested on the various ruby runtimes as well (rubinius, jruby, MRI 2.0-2.1)
Yep that's the plan!
For your golang test did you have GOMAXPROCS set to the number of CPU cores in your system?
Yep, I did. I'm sure they are some fiddly bits that could have driven it a little higher, but if you look at the tech empower latest bench, the torqbox json rack test is within spitting distance of golang, so I don't think my results are that far off.
Cool, that's pretty impressive.
Anyone willing to craft a Jubilee test implementation for us [1]? It would be nice to see the Torqbox versus Jubilee grudge match (which should echo the Undertow vs Netty title fight).
Since both Jubilee and TorqBox are easy to install, I did a quick test of both on my machine with the TechEmpower JSON app to get a feel for the relative performance - https://gist.github.com/bbrowning/8244516
My laptop gets 19k req/s with Jubilee and 118k req/s with TorqBox on the same Rack app. While Java on Vertx does fare very well in benchmarks, it's incorrect to assume this directly translates to other languages and frameworks written on top of Vertx also having the same high performance.
I guess you didn't warm up the JVM before running the tests, since the Jubilee/Vert.x stack is more heavy than the undertow counterpart, it takes longer for the JVM to apply the optimisations. I also did some benchmarks for Torqbox and Jubilee, you can find the results here: https://github.com/isaiah/jubilee/wiki/Benchmarks-against-ot...
Torqbox won the rack test, and Jubilee out performed in the rails one.
I did warm up the JVMs, actually. Those Rails numbers are far too low for TorqBox. Perhaps the best thing to do is add this to the techempower benchmark suite so a 3rd party can run and present the results.
Looks awesome. Any rails or Sinatra benchmarks ?
They are, you can find them here https://github.com/isaiah/jubilee/wiki/Benchmarks-against-ot...
How does this compare to Torquebox? When would you use one over the other?