GitHub - bkniffler/offline-sync-bench

Compare offline-first sync stacks using the same task app. The suite measures local queries, startup, edit delivery, offline recovery, conflicts, client scaling, access changes and attachments, and checks the returned data for correctness.

Includes Syncular JS/Rust, PowerSync, Turso, Zero, Electric, Electric + TanStack DB and experimental Jazz. Results describe each tested application and its guarantees.

Latest results

Latest available measurements · Apple M4 · local services · Syncular JS/Rust 0.19.0 for the 500 MB file test; 0.17.0 for other results. Latency is shown in milliseconds; lower is faster. Browser client sizes use KiB. Latency values are medians unless marked n=1; query/edit timings summarize each run’s p50. Starred entries are explained below each table. “Not supported” means the library lacks the native feature required by that test. Benchmark implementation gaps are work to fix, not product limitations.

Collection dates, configurations, sample sizes and ranges are in the linked details. Methods · Missing-case review · Failure explanations

Local task queries

Filter a task list, search titles and count tasks by group across 100,000 already-loaded tasks. Each run measures 25 operations after five warmups.

A controlled test confirmed that matching SQL indexes remove an avoidable sort. Syncular uses in-memory SQL here; PowerSync and Turso use file-backed stores. Sub-millisecond gaps are small in practice. Index investigation. Electric filters/sorts arrays; TanStack uses indexed native queries; Jazz combines indexed search with JavaScript grouping.

Workload details · Syncular/Turso details · PowerSync details · Zero details · Other client details

Queries across related records

Query 100,000 tasks across four projects. Project detail returns the first 100 tasks in one project, with each task’s title, project name and organization name. Organization dashboard summarizes all four projects with total, completed and open task counts, ordered by most open tasks.

* One run (n=1); run-to-run variability is unknown. Electric uses application lookup/order indexes over native shapes; dashboard counts are computed per query. “<0.005 ms” is below the saved p50’s 0.01 ms rounding precision; raw operation samples remain available.

** One run (n=1); run-to-run variability is unknown. Jazz uses native relationship includes for detail and materializes rows for JavaScript dashboard aggregation. Their separate costs are not isolated.

Storage matches the local-query case. Zero uses native relationships with JavaScript aggregation; their separate costs have not been measured.

Workload details · Syncular/Turso details · PowerSync details · Zero details · Other client details · Repaired case details

Starting with an empty client

Download data into a fresh client. These results use 100,000 tasks and warm services; details also cover 1,000/10,000 tasks and restarted services.

* Cold 100k startup exceeded 90 seconds; warm startup was never reached.

Screen and complete-dataset milestones are observed independently. Server storage and OS caches are retained. PowerSync compacts fixture history before timing; the earlier setup failures came from missing maintenance in this harness. Explanation. Electric and Zero load memory caches, so their “Complete local dataset” does not establish a persistent offline copy.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Reopening an offline replica

Open an existing 2,000-task store in a new process with the network blocked. Measure when the first screen and all expected rows become available.

* One earlier attempt lost its connection during initial sync; these medians use the two successful runs. The cause remains unknown.

** One run (n=1); run-to-run variability is unknown. Zero reopens its native SQLite store. A two-second preparation wait lets scheduled persistence finish before close; it is outside the timing.

*** One run (n=1); run-to-run variability is unknown. Electric reopens an application-owned SQLite cache.

The OS file cache remains warm.

Workload details · Syncular/Turso details · PowerSync details · Other client details · Repaired case details

Sharing an edit

Make 50 title edits with 200 tasks loaded on independent writer and reader clients. Measure local commit, observed server acceptance and visibility on the reader.

* Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

** The adapter deliberately disables localCommit even though the collection exposes the optimistic local update; that is not a durable queue receipt.

These milestones overlap; local commit does not prove crash durability. PowerSync’s default 1,000 ms upload throttle may contribute to its delay, but that cause remains unproven. Investigation.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Syncing edits after an outage

Queue ten writes against 2,000 tasks during a 20-second writer outage. Time queue completion and correct reader data after connectivity returns.

* Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

Queue completion and reader visibility have independent observers.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Syncing a larger offline queue

Repeat recovery with 100, 500 and 1,000 queued writes. Each column measures time from reconnection until the reader has the correct data.

* Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

The cause of the large Turso/Syncular gap remains unmeasured.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Recovering queued edits after a crash

Queue 1,000 writes, kill the writer process, reopen the same store offline, then reconnect. Verify every pending edit survives and reaches the reader.

* Memory storage explains the current skip, but IndexedDB alone does not prove immediate crash durability. Zero limits offline writes by connection state.

** Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

*** One run (n=1); run-to-run variability is unknown. The native executor restores all 1,000 transactions from an application-supplied SQLite storage adapter.

Reopen time starts at process launch; recovery times start at network restoration.

Workload details · Syncular/Turso details · PowerSync details · Other client details · Repaired case details

Two clients editing the same task

A queues an offline edit; B edits the same task online. Reconnect A and verify all three clients agree with the configured conflict policy.

* Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

The timings describe different conflict policies. Syncular rejects stale versions; PowerSync and Turso replay A’s title update. TanStack and Zero apply A’s arriving title update; Jazz retains B’s later-written field.

Workload details · Syncular/Turso details · PowerSync details · Other client details

An offline edit racing with deletion

A queues an offline edit; B deletes that task online. Reconnect A and check that the deleted task stays deleted on all three clients.

* Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

** Writer and other clients disagree after acknowledged writes and a 90-second deadline.

Syncular rejects the stale write; PowerSync and Turso’s SQL UPDATE leaves the missing row deleted. This tests one ordered race, not every conflict interleaving.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Sending one edit to many clients

With 2,000 tasks on each reader, measure one live edit reaching every connected client. The columns show time until the slowest reader is correct.

* One earlier attempt ran out of disk space during 25-reader setup; these medians use the two successful runs.

** Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

PowerSync’s observed ranges overlap at five and 25 readers; the lower 25-reader median does not establish a speedup.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Many clients reconnecting together

Disconnect five or 25 readers, accumulate 100 updates, then restore their connections together. Measure time until every reader has the complete correct dataset.

* One earlier attempt ran out of disk space during 25-reader setup; these medians use the two successful runs.

** Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

Retained storage and the maintenance pause affect the environment.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Removing access to a project

Revoke access to one of two 500-task projects. Measure removal of unauthorized rows while preserving the allowed project, both online and after reconnecting.

* The pinned local sync server exposes database replication, not per-actor row revocation while retaining other actors and server data. Lazy page fetching is not authorization filtering.

** The 60,000 ms purge deadline expires: online queries hide revoked rows but local storage retains them; reconnect also leaves queries stale. Fresh-client authorization passes.

Syncular uses explicit synchronization; PowerSync uses continuous synchronization. Local removal cannot erase previously copied data. Electric and TanStack rebuild the application cache; Zero invalidates its native memory cache. These provide different guarantees from persistent native purge.

Workload details · Syncular/Turso details · PowerSync details · Other client details

Uploading and downloading attachments

Transfer two 2 MiB objects linked to tasks. Measure upload, an uncached download and recovery after an interrupted download; verify the complete object hashes.

* One run (n=1); run-to-run variability is unknown. Early WebSocket frames are now buffered while the server session opens; upload and interrupted-download checks pass.

** One run (n=1). Uses PowerSync’s experimental native attachment queue and streaming transport with MinIO. Upload excludes file staging; download retry follows an HTTP cut after 64 KiB.

*** This library has no native attachment storage and transfer API. An application-provided uploader is outside this benchmark.

**** Electric provides read-path sync only. This benchmark requires client writes; no custom write queue or uploader is added.

***** One run (n=1). Uses Jazz’s native 256 KiB file chunks. Upload includes chunk creation and edge persistence; retry follows a disconnect after the first chunk and reuses the native cache. These boundaries differ from the object-store clients.

Syncular and PowerSync retry object-store downloads; Jazz reads native synced chunks. The footnotes explain the different upload and retry boundaries.

Workload details · Syncular JS details · Syncular Rust details · PowerSync and Jazz details

Uploading and downloading a 500 MB file

Upload one 500,000,000-byte file linked to a task, then download it in a new process with an empty client cache. Upload includes native staging; download ends when complete bytes are materialized. Verify the full SHA-256 hash. Syncular reports medians of three controlled runs (n=3); PowerSync and Jazz retain single runs (n=1). All use local services; file preparation and final hash validation are outside the clock.

* Syncular JS/Rust 0.19.0: medians of all three runs per client (n=3), collected in controlled alternating pairs against 0.18.0 under the same 0.19.0 server. Upload includes staging, transfer and metadata acceptance.

** PowerSync uses its experimental native attachment queue and filesystem transport. Its retained single run (n=1) excludes the final metadata-acceptance wait, so it has a different stopping point from Syncular.

*** These libraries have no native attachment upload/download feature; Electric is read-only.

**** Jazz retains a single run (n=1) using its default 256 KiB chunks (1,908 parts). Its native helper awaits each part insertion; the other measured clients transfer whole objects through MinIO.

Workload, cached fixture and raw results

Browser client size

Initialize a real browser client and verify local storage across a reload. Count the requested JavaScript, workers and WASM—including embedded WASM—once per file. JavaScript is minified; gzip totals use level 9. These are storage-ready startup sizes. Each cell shows raw/gzip KiB. Core includes SDK code and shared adapters; Storage includes separate engine loaders, workers and WASM. Integrated storage code stays in Core. 1 KiB = 1,024 bytes.

* Syncular Rust and Turso use native-host clients in this harness. PowerSync uses its Web SDK for this comparison.

** Plain Electric is an in-memory, read-only client and refetches after reload; it is not an equivalent persistent offline client.

*** Jazz’s WASM contains both storage and sync logic; its Storage figure is not a pure database size. TanStack includes native SQLite persistence and an IndexedDB outbox; its worker’s embedded WASM is counted once.

Browser checks and complete asset inventory · Scope and configurations

Staying responsive while syncing

A browser page keeps working while its client syncs. The controller types a key every 100 ms and the page refreshes a 50-row task list four times a second. The client first downloads 100,000 tasks into an empty store (initial sync). The page then closes, the server commits 20,000 task updates, and the reloaded page reopens the same store and catches up (catch-up). Plain Electric keeps no local store, so its catch-up downloads every task again. Keystroke p95 is the 95th-percentile time from a key event to the next frame; a responsive page stays under one frame (16.7 ms). Main thread blocked is the long-animation-frame blocking time as a share of the phase. Medians of 3 trials per client, each in a fresh Chromium profile.

* Syncular Rust and Turso use native-host clients in this harness; there is no browser client to measure.

** Jazz has no browser adapter for this case yet; its native 100,000-task startup did not complete within 90 seconds.

On a slower device. The same trials with every Chromium process except the GPU at background CPU priority. DevTools CPU throttling is not used because it does not slow workers. The slowdown column is the measured cost of a fixed loop on the main thread and in a worker, relative to default priority.

Workload, conditions and raw trials · Definition

Run a benchmark

Install Bun and start Docker, then:

bun install --frozen-lockfile
bun run bench:run -- --stack syncular --scenario local-query

The harness resets the selected stack’s benchmark fixtures. Running campaigns and publishing results · Benchmark definitions