Kawipiko – fast static HTTP server in Go
github.comThe fact it is using three different entirely HTTP libraries to be able to support HTTP/1, 2 and 3 is kind of telling on the complexity of modern HTTP.
[Kawipiko's author here] The main HTTP "engine" behind Kawipiko is `fasthttp`, <https://github.com/valyala/fasthttp> that supports only HTTP/1.1 (with or without TLS). However for experimentation I've also integrated HTTP/2 and HTTP/3 support, although the performance isn't that great...
In fact, there is one ticket open on `fasthttp` about HTTP/2 support, however (as I've also commented on this issue), I don't think that's the way to go for the intended use-cases: <https://github.com/valyala/fasthttp/issues/144>
My current opinion is this: HTTP/2, and especially HTTP/3, are mainly meant for CDN's and highly performant HTTP load-balancers (like HAProxy), meanwhile HTTP/1.1 is the best solution for the backend.
This looks like an fun project, but I fail to see where it could be useful outside of a proof of concept.
The focus is on the performance, yet "kawipiko is at least on-par with NGinx". NGinx does require file open-read-close, but with the OS cache, I think the cost is very small. And Nginx is much more compliant, battle-tested and versatile. For example it can send the right pre-compressed file that suits the Accept-encoding header of the request, something that kawipiko does not support.
So I guess the selling feature is that the web server is just a single static executable. It's probably useful in some cases, but I can't imagine them, even for a doc server embedded in hardware.
[Kawipiko's author here] NGinx is great at what it does, however in order to get good performance out of it one must also be a master of its configuration knobs, and knowing all of their implications. For example while benchmarking NGinx I've observed quite different performance characteristics for simple patterns such as asking for `/some-folder/` that fallbacks to `/some-folder/index.html` vs explicitly asking for `/some-folder/index.html`.
On the other hand with Kawipiko there aren't many configuration knobs; it is already tuned for a single particular use-case: serving static content for websites (with small resources).
Also, a particular use-case where Kawipiko can shine over NGinx, is for a website with lots of small resources (as with my Leaflet tiles example). NGinx can't decide where those resources are placed on disk, thus the OS file-system caching and read-ahead can't help too much here. Meanwhile with Kawipiko, where everything is bundled together, it receives a lot of help from the OS in this regard.
As with regard to the "battle-tested and versatile", Kawipiko uses <https://github.com/valyala/fasthttp> as its HTTP engine, which itself is used in quite a lot of Go-based products, thus one could state that it's "battle-tested enough". :)
With regard the "even for a doc server embedded in hardware", why don't you think Kawipiko is a good fit in that case? (In fact this was one of its main use-cases.)
BTW, I've just tried running Kawipiko on an Android tablet (ARM64) to export the same demo as available at the link in the repository, and it works great!
In fact, benchmarking it over wireless + Wireguard with 128 concurrent connections yielded around ~2K requests/second and the overall CPU stayed under 1% (for the server process). (Granted the main issue was latency, but the average was well under 25 ms.)
Thus for a portfolio site that you take with you at conferences / meetings it works very well.
So from a quick glimpse, everything is served as immutable with 60 minutes max-age? So swapping out the cdb, will take 60 minutes for client caches to be fully updated?
The `Cache-Control` header could be configurable (although at the moment the Kawipiko archiver hard codes the 60 minutes max age, and the server just uses the headers present in the archive). (In fact there is a flag to disable setting this header.)
On the other hand, for the intended use-cases (mostly static contents behind a CDN), a 60 minute caching period is still acceptable and thus the default.
> does not support per-request decompression / recompression;
this sound like lots of headache as it doesn't support `Accept-Encoding` header.
At the moment the site operator has three choices:
* bundle two separate archives, one with compression and one without (provided it fits in 4 GiB), and use something like HAProxy to choose between the two backends based on the Accept-Encoding header;
* (most likely) use a CDN service (like CloudFlare) that would do caching and re-encoding based on what the actual client supports;
* choose to just support compressed contents, based on the next observation;
Any "modern enough browser" (say since 2018, and here I include things like Netsurf), does support even Brotli compression, let alone gzip. Indeed Lynx (the console browser) doesn’t support Brotli, but it still does support gzip.
Thus, for all "real clients" out there, serving straight compressed contents is not an issue, thus one could choose to just ignore the Accept-Encoding header.
> the static content is served from a CDB file with almost no latency
That is cool! I use CDB in some projects and love it.
What's CDB? A quick search reveals it's a SAP's data archive format? (not sure how it relates to an HTTP server)
Useful for proof of concepts & private content. I'm wondering, for public static content, do anyone use webservers these days when CDNs are available as a service?
Well, as the CDN initials hint (Content Delivery Network), these services only "deliver" your content; the content must still live somewhere...
That "somewhere" can either be another cloud service dedicated to static sites such as Netlify, GitHub Pages, CloudFlare Pages; a general purpose storage service such as AWS S3; a "serverless" service running on something like AWS Lambda or Fly.io; or even a self-hosted Linux server (running on hardware or as a VM in a IaaS provider such as Linode).
Kawipiko tries to solve the last two scenarios, when one wants to serve the content from a self-hosted VM.
How does this compare to Caddy?
[Caddy maintainer here] It's probably faster, because it focuses on only serving static content. Caddy is designed to be general purpose. Caddy uses net/http (Go's stdlib implementation), Kawipiko uses "fasthttp" which is a separate library that should be more performant for simple uses.
But of course, do your own benchmarking, for your own usecase, and see which one works best for your needs. IMO benchmarking webservers is usually misleading because everyone has slightly different needs and since webservers are designed to be configurable.
All that said, with the upcoming release, Caddy will support being built with an embedded filesystem with https://pkg.go.dev/embed so your static content can ship with your Caddy binary and get served from memory. We haven't done much testing to see how fast this is in general, but it should be pretty fast in theory.
The main problem with Go when it comes to raw performance is heap memory allocation. `fasthttp` (and the rest of Kawipiko's code on the main serving path) does all it can to avoid any sort of memory allocation (or "heap escape").
On the other hand, Go's `net/http` uses quite a bit of heap allocation, thus it will never beat `fasthttp` in terms of raw performance.
That being said, the numbers are pointless... From what I remember Go's `net/http` is able to serve a few tens of thousands of requests per second, thus it won't be the main bottleneck.
> The main problem with Go when it comes to raw performance is heap memory allocation. `fasthttp` (and the rest of Kawipiko's code on the main serving path) does all it can to avoid any sort of memory allocation (or "heap escape").
How do they manage to do that? One aspect I can think of is, trying not to use pointers and data structures with dynamic size
> How do they manage to do that? One aspect I can think of is, trying not to use pointers and data structures with dynamic size.
On the contrary, they do use lot's of pointers (actually byte slices) and structures with dynamic size, however they rely heavily on buffer pooling and carefully appending to such byte buffers instead of allocating new ones.
[Kawipiko's author here] Caddy, and Nginx, Apache, etc., are general purpose web servers that either serve content directly from the file-system or reverse proxy requests to other backends. Meanwhile Kawipiko has only one purpose: serve resources from an embedded key-value database, namely the venerable CDB <https://cr.yp.to/cdb.html>.
While general purpose web servers use open-(read+send)*-close syscalls (and other tricks such as `sendfile`) for each request, Kawipiko uses a single memory-mapped database for all requests, thus all the I/O is mediated (and cached) by the OS.
Thus, one could say that Kawipiko serves a well defined niche, heavily optimizing for that particular use-case, meanwhile the other servers although can do lots of other things, they can't be optimized for any of those use-cases.
I guess I'm curious how all the theoretical benefits pan out in the end and if the benefit is worth the separate project.
For example in one production deployment, Kawipiko is used to serve a snapshot of a former dynamic web-site that is now conserved. That snapshot has around 600K files and would eat around 12 GiB of storage; but with Kawipiko all that is bundled to just one single file of ~3.5 GiB that can easily be moved around and deployed essentially anywhere.