Programming at Light Speed: Building the Gravity Query Engine
pgrust.com > Normally, when engineers try to make software faster, they work from the bottom up. They run a benchmark, look at a flamegraph, find a hotspot, and figure out how to speed up their code by 15% or 20%. The problem with working from the bottom up is that you don't know what the ceiling is. You might be 20% away from the fastest possible code the hardware can run, or you might be 100x away.
> When we started building Gravity, we flipped this question around. We asked ourselves: how fast can a database actually be? Instead of incrementally speeding up our existing query engine, we wanted to start from the physical speed limit of our hardware and file format and work backward. I like to call this “programming at light speed”.
See for example: https://www.marginalia.nu/log/a_106_native_calls/
Apologies for being nitpicky, but I felt it might be useful for other readers to contextualize that this part of the approach is not novel. Their use of Claude to generate those optimized benchmarks is interesting enough already, so I don’t feel that calling extra attention to this part in their write up is necessary.
(I authored that blog post)
I think it kinda depends on the stage of the development. If you have an application already, flame graph whack-a-mole is the normal starting point. This is the correct way to go about it because of Amdahl's law. Random optimization is extremely unlikely to pay off in a way that leads to measurable application wide performance improvements; no matter how big improvements you find for a function that's 0.1% of the run time, you'll never see more than a 0.1% improvement of the run time.
Bottoms up optimization is what you reach for if you have a known hot spot (from a profiler flame graph), or you're deliberating design choices when building something from scratch (where you can be reasonably sure performance will matter).
Of course all of this is exponentially harder for database-adjacent work since you have os and hardware level caches, possibly a buffer pool of your own, and in production you usually operate in fickle semi-resident states that are extremely hard to replicate in a benchmark scenario. So you end up with a lot of benchmarks (cold cache, hot cache, cgroup memory constraints) hoping that sort of fixes it.
You’re absolutely right. Re-reading my take I recognize it is unfair to present these as “naïve”/“serious”. Optimization is a really complex area and there’s a lot of nuance and layers on the techniques involved.
By the way, thanks for marginalia and the great write-ups on your blog.
After the recent somewhat discouraging stuff with clickbench that made the news, I must say this blog post is very cool.
Sounds like you found an excellent way of structuring this problem to solve it with Claude (for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly), and the writeup is very interesting and readable too!
Kudos, looking forward to more blog posts like this one.
> for those who don't click into the article - writing hand/claude-optimized Rust code for each query, checking its assembly, and trying to make their own optimizer generate something close to that same assembly
Invaluable summary, thanks.
Are the pgrust guys lisper historically ?
Yep! My username is a portmanteau of my last name and lisper.
My first handful of blog posts were all about doing really cool stuff in Lisp: https://malisper.me/building-fizzbuzz-fractran-bottom/
ahh right, I didn't read the github project owner, now it all make sense :)
Not answering your question directly, but FWIW, that’s the default IR notation Claude reaches for.
oh really ? I never poked at that kind of problems using claude. still interesting that claude's training placed lisp at the top for IR stages.. even though historically it's obviously one of the culture that worked on interpretation and compilation the most, it's not mainstream. But I guess for sql engine teams, it decided that it was perfectly on point.
I found that out in the context of imperative-style language development. I wanted the IR to be as commands with structs and it took some effort to steer it away from Lisp.