Show HN: Ayder HTTP-native durable event log in C (50K msg/s, P99 3ms, Raft HA)

2 points by Aydarbek 3 months ago · 1 comment · 1 min read

Reader

Hi HN — I built Ayder, a single-binary durable event log / message bus in C. The idea: what if event streaming was just HTTP?

  # Produce
  curl -X POST 'localhost:1109/broker/topics/orders/produce' -d '{"item":"widget"}'
  
  # Consume  
  curl 'localhost:1109/broker/consume/orders/mygroup/0?encoding=b64'

No JVM, no ZooKeeper, no thick client libraries. Just HTTP.

What it does: append-only log with per-partition offsets, consumer groups, durable writes via sealed AOF + crash recovery, HA with Raft consensus (3/5/7 nodes), leader redirects, idempotent produce, KV store with CAS + TTL.

Benchmarks (real network, 3-node Raft, DigitalOcean 8 vCPU, wrk2, 64B payload):

  Throughput: 50,000 msg/s sustained
  P50: 1.58ms | P99: 3.35ms | P99.9: 8.62ms

Server-side P99.999 is 1.2ms — the tail you see in benchmarks is network/kernel, not the broker.

Curious finding: real NIC beats loopback at high concurrency (P99 3.31ms vs 5.79ms). Makes sense — loopback means client and server fight for CPU cores.

I'd love feedback on: API shape, HA ergonomics, what you'd expect from "durable event log over HTTP" in 2025.

Happy to discuss implementation details — Raft catch-up, AOF format, zero-copy parsing, whatever interests you.

AydarbekOP 3 months ago

Author here. Happy to answer questions about the implementation.

A few details that might interest HN: - Raft replication handles 50K msg/s with sync-majority writes - HTTP parsing is 0.4ms at P99.999 (picohttpparser, zero-copy) - Tested durability with SIGKILL — recovery replays sealed AOF

What would you want from an HTTP-native event log?

Settings

Show HN: Ayder HTTP-native durable event log in C (50K msg/s, P99 3ms, Raft HA)

Keyboard Shortcuts