Honker

honker adds Postgres-style NOTIFY/LISTEN semantics to SQLite, with a durable pub/sub, task queue, and event streams on the side, without client polling or a daemon/broker. Cross-process wake latency is ~0.7 ms p50 on an M-series laptop.

In its basic form it’s a plain SQLite loadable extension, so any language that can SELECT load_extension('/path/to/libhonker_ext') gets the same queue, streams, and notifications on the same file. Bindings for Python, Node, Rust, Go, Ruby, Bun, Elixir, C++, .NET / C#, Java, and Kotlin share one on-disk format.

SQLite is backing real work now — Bluesky’s PDS, Fly’s LiteFS, Turso, weekend projects that somehow ended up in production. Once real work flows through a SQLite-backed app, you need a queue. The usual answer is “add Redis + Celery.” That works, but introduces a second datastore with its own backup story, a dual-write problem between your business table and the queue, and the operational overhead of running a broker.

honker takes the approach that if SQLite is the primary datastore, the queue should live in the same file. That means INSERT INTO orders and queue.enqueue(...) commit in the same transaction. Rollback drops both. The queue is just rows in a table with a partial index.

• Cross-process NOTIFY/LISTEN-style signaling on a single SQLite file, without Redis or a broker
• Durable work queues with retries, priority, delayed jobs, expiration, and dead letters
• Transactional outbox patterns where your business write and enqueue commit together or roll back together
• Durable streams with per-consumer offsets
• Time-trigger scheduling with a leader-elected scheduler: 5-field cron, 6-field cron, and @every <n><unit>
• Named locks, rate limiting, and opt-in task result storage
• Thin cross-language bindings over the same extension-owned SQLite layout

Enqueue atomically with a business write, then consume. Same .db file, same on-disk format, now including Java and Kotlin.

import honker
db = honker.open("app.db")
q = db.queue("emails")
# Enqueue in the same transaction as the business write.
with db.transaction() as tx:
    tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99])
    q.enqueue({"to": "[email protected]", "order_id": 42}, tx=tx)
# Worker wakes on database updates or due deadlines.
async for job in q.claim("worker-1"):
    await send_email(job.payload)
    job.ack()

@q.task(retries=3, timeout_s=30)
def send_email(to, subject):
    ...
    return {"sent_at": time.time()}
r = send_email("[email protected]", "Hi")   # enqueues, returns TaskResult
print(r.get(timeout=10))                    # blocks until worker runs it

honker polls SQLite’s PRAGMA data_version every millisecond. That’s a monotonic counter SQLite increments on every commit from any connection, journal mode, or process — a ~3 µs read for a precise wake signal. A background thread fans the tick out to every subscriber, which runs SELECT ... WHERE id > last_seen and yields new rows. One poller thread per database regardless of subscriber count.

Idle cost is that one lightweight SELECT per millisecond per database — no page-cache pressure, no writer-lock contention, no kernel file watcher in the mix. Listener count scales for free because the wake signal is one shared poll, not one query per listener.

Advanced users can opt into experimental kernel filesystem-event or mmap WAL-index backends; see Watcher backends.

The queue, stream, and pub/sub primitives are all INSERTs into tables managed by the extension. Calling queue.enqueue(payload, tx=tx) inside your business transaction means the job row is ACID with the INSERT INTO orders that preceded it. Rollback drops the job along with everything else.

Honker intentionally does not support SQLite in-memory database filenames (:memory:, file::memory:?cache=shared, or file:<name>?mode=memory&cache=shared). Bare :memory: is per connection, and SQLite’s shared-memory URI forms only share state inside one process. For production-faithful tests, use a temporary file-backed .db; it preserves the same locking, wake, crash/reopen, and multi-process semantics as production.

pg_notify gives you fast cross-process triggers but no retry or visibility. Huey is the SQLite-backed Python task queue honker draws the most from. pg-boss and Oban are the Postgres-side gold standards. If you already run Postgres, use those.