GitHub - t4db/t4: S3-durable key-value storage with etcd v3 compatibility

An embeddable, S3-durable key-value store for Go, with an etcd-compatible standalone server.

Embedded-first — t4.Open(cfg) is the entire API. No sidecar, no daemon.
S3-durable — WAL segments and periodic checkpoints are uploaded to S3. A node that loses its disk recovers automatically.
Multi-node — Leader elected via an S3 lock. Followers stream the WAL in real time and forward writes transparently.
etcd v3 compatible — The standalone binary speaks the etcd v3 gRPC protocol, including multi-key transactions.
Twelve-factor config — CLI flags can be supplied through T4_* environment variables.
Branches — Fork a database at any checkpoint with zero S3 copies. Each branch writes to its own prefix; shared SST files are deduplicated automatically.

Embedded usage

import "github.com/t4db/t4"

node, err := t4.Open(t4.Config{
    DataDir: "/var/lib/myapp/t4",
})
defer node.Close()

rev, err := node.Put(ctx, "/config/timeout", []byte("30s"), 0)

kv, err := node.Get("/config/timeout")
fmt.Println(string(kv.Value)) // 30s

events, _ := node.Watch(ctx, "/config/", 0)
for e := range events {
    fmt.Printf("%s %s=%s\n", e.Type, e.KV.Key, e.KV.Value)
}

With S3 durability

import (
    "github.com/t4db/t4"
    "github.com/t4db/t4/pkg/object"
)

store, err := object.NewS3StoreFromConfig(ctx, object.S3Config{
    Bucket: "my-bucket",
    Prefix: "t4/",
    Region: "us-east-1",
    // Endpoint: "http://localhost:9000", // MinIO or another S3-compatible store
})
if err != nil {
    return err
}

node, err := t4.Open(t4.Config{
    DataDir:     "/var/lib/myapp/t4",
    ObjectStore: store,
})

Standalone binary

The t4 binary exposes the etcd v3 gRPC protocol. Use etcdctl, the official Go client, or any other etcd v3 compatible tool.

go install github.com/t4db/t4/cmd/t4@latest

# Single node, local only
t4 run --data-dir /var/lib/t4 --listen 0.0.0.0:3379

# Single node with S3
t4 run --data-dir /var/lib/t4 --listen 0.0.0.0:3379 \
           --s3-bucket my-bucket --s3-prefix t4/

# The same configuration can come from environment variables.
T4_DATA_DIR=/var/lib/t4 \
T4_LISTEN=0.0.0.0:3379 \
T4_S3_BUCKET=my-bucket \
T4_S3_PREFIX=t4/ \
T4_S3_REGION=us-east-1 \
t4 run

# Verify
etcdctl --endpoints=localhost:3379 put /hello world
etcdctl --endpoints=localhost:3379 get /hello

For offline inspection of a local data directory, use t4 inspect:

# Show local metadata without starting a server.
t4 inspect meta --data-dir /var/lib/t4

# Explore current keys.
t4 inspect list --data-dir /var/lib/t4 --prefix /config/
t4 inspect get --data-dir /var/lib/t4 /config/timeout

# Explore revision history and changes over time.
t4 inspect history --data-dir /var/lib/t4 /config/timeout
t4 inspect diff --data-dir /var/lib/t4 --from-rev 100 --to-rev 120 --prefix /config/

Multi-node and production setup: see Operations.

Branching

Branches fork a database from an existing S3 checkpoint without copying shared SST files.

# Register the branch against the source prefix.
checkpoint_key=$(t4 branch fork \
  --s3-bucket my-bucket \
  --s3-prefix t4/ \
  --branch-id experiment)

# Start the branch in its own prefix, using the source prefix as its ancestor.
t4 run \
  --data-dir /var/lib/t4-experiment \
  --listen 0.0.0.0:3379 \
  --s3-bucket my-bucket \
  --s3-prefix t4-experiment/ \
  --branch-prefix t4/ \
  --branch-checkpoint "$checkpoint_key"

When the branch is retired, remove its registry entry so future GC can reclaim unneeded source objects:

t4 branch unfork --s3-bucket my-bucket --s3-prefix t4/ --branch-id experiment

Documentation

Full documentation is available at t4db.github.io/t4.

Document	Contents
Getting Started	Quickstart for standalone server and embedded Go library
API Reference	Full Go API — methods, types, errors, branching
Configuration	All config fields and CLI flags
Operations	Multi-node clusters, S3, TLS, authentication, RBAC, observability
Backup and Restore	Checkpoints, point-in-time restore, branching, retention
Security	TLS, mTLS, client auth, RBAC setup
Recipes	Distributed locks, service discovery, common patterns
Kubernetes	Helm chart, StatefulSet deployment
Docker Compose	Local, MinIO-backed, and multi-node cluster examples
Architecture	Internals — WAL, checkpoints, leader election, replication
Benchmarks	T4 vs etcd benchmark results and analysis
Migrating from etcd	Compatibility table and migration steps
Troubleshooting	Diagnostics, debug logging, and common fixes
FAQ	Frequently asked questions