Start boring.
A normal put writes a typed raw envelope through SlateDB, so direct engine reads do not pretend the bytes are anything else.
Transcendental compaction
A data-free key-value store. Pi for storage. Regret for reads.
PiLSMer writes your data normally, then uses planning or SlateDB compaction to replace stored values with instructions for finding equivalent byte sequences inside a deterministic stream. Reads still work. Everything else gets worse.
What changes on disk?
The wrapper can turn a raw value into a plan: stream identity, offsets, chunk lengths, and hashes. That plan is enough to reconstruct the original bytes while making the storage engine look worse at its job.
Find the bytes elsewhere.
The planner searches deterministic stream prefixes and replaces the value with a reconstruction recipe when it can.
Return the same bytes.
Gets reconstruct the logical value from the plan, verify the hash, and hand back the original payload.
One tiny humiliation loop.
The useful demo path writes a JSON invoice, asks PiLSMer to plan it, and then reads the exact value back. The page stays on the mechanism and leaves performance claims for later data.
pilsmer receipt
$ cargo run -q --bin pilsmer -- init /tmp/pilsmer-demo/db $ printf '{"total":49.99,"status":"paid"}' > /tmp/pilsmer-demo/invoice.json $ cargo run -q --bin pilsmer -- put /tmp/pilsmer-demo/db invoice:123 /tmp/pilsmer-demo/invoice.json $ cargo run -q --bin pilsmer -- plan-key /tmp/pilsmer-demo/db invoice:123 planned: invoice:123 $ cargo run -q --bin pilsmer -- get /tmp/pilsmer-demo/db invoice:123 {"total":49.99,"status":"paid"}
Compaction into nonexistence.
The moving pieces are deliberately normal: envelopes, deterministic streams, a planner, reconstruction, and a SlateDB wrapper. The bad idea lives in the semantics.
Frame every value.
Raw bytes and planned values are encoded before they reach the underlying store.
Index deterministic bytes.
Pi, e, sqrt(2), and the SHA-256 counter stream give the planner places to point.
Swap payload for regret.
Planning, vacuuming, or compaction can replace a raw envelope with metadata that locates the same logical bytes.
Reconstruct before returning.
The read path rebuilds bytes from the stream and checks the logical hash before returning user data.
Where the joke becomes code.
The implementation is small enough to inspect directly. These links are the interesting seams.