Author: Nikhil Ranjan - [email protected] - github.com/niklabh
Implementation: https://github.com/niklabh/velocidb
1. The Case for Architectural Revolution
1.1. Context and Goals: Moving Beyond File-Based Isolation
The original SQLite achieved monumental success by adhering to principles of simplicity, zero-configuration, and universal portability, implemented using the C language for maximal performance and minimal dependencies. Its foundational concurrency model—relying primarily on operating system file locks (such as the EXCLUSIVE lock during write transactions) managed by the Virtual File System (VFS)—was robust and ensured crash safety. However, this original design, conceived decades ago, is fundamentally mismatched with contemporary hardware architectures.
Modern commodity systems, from embedded devices to multi-socket servers, are dominated by multi-core processors, NVMe storage providing high IOPS (I/O Operations Per Second), and, increasingly, Persistent Memory (PMEM). In this environment, performance bottlenecks have fundamentally shifted. They are no longer dictated by magnetic disk latency, but rather by CPU core contention, I/O path inefficiencies, and the significant latency introduced by context switching and kernel overhead. The traditional, blocking, synchronous I/O model of classic SQLite severely underutilizes modern multi-core parallelism, especially when concurrent writes are attempted.
The primary goal of designing a modern embedded database (referred to as VelociDB) is to preserve the core philosophy—zero administration, single-file deployment, and cross-platform compatibility—while transforming its internal architecture. This transformation must migrate the system from a single-process, pessimistic-locking, synchronous I/O model to a high-throughput, internally concurrent, asynchronous execution model. This necessitates foundational changes across the entire stack: from the implementation language to the concurrency control mechanism, and the on-disk data structures.
1.2. Key Pillars of the Modern Embedded Database
The architectural blueprint for VelociDB rests on five foundational shifts required to leverage current hardware capabilities:
Language and Safety: Transitioning from C to Rust to guarantee memory and thread safety during the implementation of complex, concurrent internals.
Concurrency: Replacing file-level locking with Multi-Version Concurrency Control (MVCC) and lock-free structures to enable concurrent reads and writes, scaling effectively across multi-core processors.
I/O Stack: Rebuilding the VFS/Pager around native asynchronous I/O (AIO/io_uring) and direct access (DAX) to high-speed NVMe and Persistent Memory.
Execution: Converting the Virtual DataBase Engine (VDBE) from scalar to vectorized execution, leveraging Single Instruction, Multiple Data (SIMD) units for query acceleration.
Synchronization: Integrating Conflict-Free Replicated Data Types (CRDTs) to support robust bi-directional synchronization and operation in modern, often disconnected, hybrid cloud/edge environments.
2. Foundational Choice: Implementation Language and Ecosystem
2.1. Analysis of C's Legacy vs. Modern Requirements
The original decision to implement SQLite in generic C was strategic and justified at the time of its inception in 2000. C provided unparalleled performance, stability, and, critically, universal compatibility with minimal runtime dependencies. Furthermore, contemporary alternatives like C++ and Java were perceived as immature, suffering from compiler inconsistencies and growing pains, making C the safer choice for a robust library.
However, this legacy is now the greatest constraint on architectural evolution. As the system programming landscape has matured, C's primary weakness—manual memory management—has become a significant liability. Building the complex concurrency primitives required by modern hardware, such as MVCC and lock-free data structures, is highly challenging and inherently fragile in C, demanding developers maintain low-level control over memory and synchronization without compile-time guarantees. The risk of introducing security vulnerabilities (like buffer overflows) or internal data races in a globally deployed library outweighs the perceived benefit of C simplicity.
2.2. The Mandate for Memory Safety and Concurrency: Adopting Rust
The implementation language for VelociDB must be Rust. Rust is explicitly designed to offer the low-level control and performance characteristics comparable to C or C++, but mandates stringent safety guarantees enforced at compile time.
2.2.1. Compile-Time Guarantees (Ownership and Borrowing)
Rust enforces memory safety primarily through its unique ownership and borrowing model. This system eliminates the class of vulnerabilities stemming from dangling pointers, double-frees, and buffer overruns. Crucially for a concurrent database engine, Rust’s ownership system, combined with the Send and Sync traits, guarantees thread safety and prevents data races at compile time. This mechanism is paramount, as the new architecture relies heavily on complex, highly concurrent internal components (e.g., lock-free queues and MVCC version chains). This system allows the system architect to confidently implement advanced concurrency features that would be exceptionally difficult and error-prone to achieve safely in C or C++. While Rust provides raw pointers for necessary low-level interactions (such as FFI or specific hardware access), the vast majority of the codebase benefits from memory protection.
2.2.2. Performance and Zero-Cost Abstractions
Rust is fundamentally a systems programming language that targets execution speed. Unlike languages that rely on runtime garbage collection (GC), Rust's memory management is handled through compile-time ownership rules, resulting in zero-cost abstractions. This ensures that VelociDB maintains the "blazing-fast" performance characteristics of its predecessor while incorporating safer, more complex internal logic. This low-level efficiency is critical for a high-performance embedded library.
2.2.3. Retaining Portability (C FFI and Static Linking)
One of SQLite's core strengths is its ability to be integrated into nearly all systems that can call a C library. To retain this crucial feature, VelociDB, written in Rust, must provide a clean Foreign Function Interface (FFI). Rust supports compilation targets that produce C-compatible libraries (cdylib for dynamic linking or staticlib for static linking). Functions exposed to external languages would use the pub extern "C" signature and the #[no_mangle] attribute to ensure a well-defined symbol linkage.11 This approach maintains the library’s universal compatibility while isolating the performance-critical core in a safe environment.
2.2.4. Integrated Asynchronous Runtime
The shift to asynchronous I/O requires a high-performance runtime. Rust’s ecosystem provides robust, specialized asynchronous runtimes, such as Tokio, which offer low-level I/O, timer, and scheduling facilities. This native support for async/await primitives simplifies the implementation of non-blocking I/O operations, which is essential for maximizing CPU utilization when dealing with microsecond-latency storage devices.
Feature: Memory Management
Classic SQLite (C): Manual (malloc/free). High risk of safety and security issues.
Modern
Embedded DB (Rust): Ownership/Borrowing system; Compile-time safety guarantees.
System Impact: Eliminates high-risk memory vulnerabilities, vital for a globally used library.
Feature: Concurrency Model
Classic SQLite (C): Externalized via VFS file locks. Internal multi-threading is difficult/fragile.
Modern Embedded DB (Rust): Built-in data race prevention (Send/Sync traits). Native async runtime support (Tokio).
System Impact: Prerequisite for multi-core scaling and non-blocking I/O.
Feature: Dependency Footprint
Classic SQLite (C): Minimal standard C library dependency.
Modern Embedded DB (Rust): Low-dependency static compilation possible, leveraging modern crates.
System Impact: Retains core portability; C FFI allows easy library linkage.
3. Re-architecting the I/O Stack for Persistent Storage (The Modern Pager)
The design of the Pager and VFS layers represents the most critical divergence from the original SQLite architecture, driven by the emergence of high-speed NVMe and Persistent Memory (PMEM). The traditional focus on abstracting slow, high-latency mechanical disks is replaced by an optimization mandate focused on minimizing CPU stalls and maximizing data locality across a tiered storage hierarchy.
3.1. The New VFS: Abstraction for NVMe and Cloud Storage
3.1.1. High-Performance NVMe Exploitation
Modern NVMe SSDs, leveraging PCIe interfaces, can deliver millions of IOPS and bandwidths exceeding 7 GB/s per device. Yet, existing out-of-memory database systems often fail to exploit this potential, achieving only a fraction of the hardware capability due to synchronous I/O overhead and excessive kernel interaction. The architectural conclusion is that when storage devices become this fast, software overhead becomes the dominant bottleneck.
3.1.2. Native Asynchronous I/O Integration
The modern VFS must therefore be fundamentally asynchronous. It cannot rely on blocking kernel calls that force threads to sleep while waiting for I/O completion. Instead, it must utilize advanced kernel facilities such as Linux's io_uring or Kernel-Asynchronous I/O (KAIO) where available. The Rust-based VFS abstracts these platform-specific mechanisms using its native async/await primitives, managed by the Tokio runtime. This design allows a thread requesting an I/O operation to yield control back to the scheduler, enabling the core to process hundreds or thousands of other tasks (e.g., executing other transactions' bytecode) while waiting for the I/O to complete. This approach maximizes multi-core utilization and overall throughput.
3.1.3. Decoupling Query and Storage Engines
Implementing native asynchronous I/O necessitates a subtle but profound change in the execution pathway. In the original design, the VDBE (Virtual DataBase Engine) executes bytecode instructions, tightly coupled with the Pager/B-Tree layer for data access. In VelociDB, the query engine must be conceptually decoupled from the storage engine. Any VDBE bytecode instruction requiring data that is not immediately present in the memory cache must trigger an asynchronous I/O instruction. This allows the executing thread to yield back to the scheduler without blocking, preventing long tail latencies and promoting better responsiveness, particularly crucial in serverless or edge computing contexts.
3.2. Integrating Persistent Memory (PMEM/NVM)
Persistent Memory (PMEM), such as Intel Optane DC, offers persistence combined with latency approaching DRAM speeds.17 This new tier of storage provides byte-addressability and persistence, fundamentally blurring the traditional separation between volatile RAM and non-volatile disk.
3.2.1. DAX Mode and OS Page Cache Bypass
To properly exploit PMEM, the VFS must support Direct Access (DAX) mode.18 DAX allows applications to bypass the kernel’s page cache and perform direct memory mapping (mmap(2)) to the PMEM media. This bypass eliminates the overhead of copy operations and kernel synchronization that define traditional I/O paths. Utilizing DAX requires specific OS setup, typically mounting a DAX-capable file system (like XFS or NTFS) using the dax mount option, operating on PMEM devices configured in fsdax mode. Optimizations related to DAX often involve ensuring memory and extent alignments, commonly 2MB, to match processor page sizes, which further improves performance.
3.2.2. PMEM-Optimized Logging and Journaling
Classic SQLite achieves crash safety through Write-Ahead Logging (WAL) or rollback journals, which rely on slow, latency-intensive fsync operations to ensure data durability on disk. With PMEM, the crash-safety mechanism can be redesigned. PMEM-optimized log structures, such as those provided by libpmemstream, allow the transactional redo/undo log to be written directly to persistent, byte-addressable memory. This persistence is immediate upon write, dramatically reducing the latency associated with transaction commits, effectively replacing expensive disk flushes with fast memory writes. This capability is instrumental for handling high-frequency, low-latency embedded transactions.
3.3. Memory Mapping ($mmap$) Revisited
The use of memory-mapped files ($mmap$) is beneficial for data processing due to its low-latency access and efficiency, potentially employing zero-copy mechanisms to operate directly on data. VelociDB would utilize $mmap$ where available for the data file, as it effectively offloads page management to the operating system's virtual memory subsystem.
However, $mmap$ usage introduces complexities, particularly in concurrent, multi-process environments. If memory is mapped at different virtual addresses in different processes, direct pointers cannot be stored; instead, all internal references must rely on offsets relative to the base address of the mapping. Furthermore, in modern systems with Non-Uniform Memory Access (NUMA), accessing memory segments mapped on a distant NUMA node can incur significant speed penalties. The VFS layer must therefore encapsulate all $mmap$ usage, ensuring that internal data structures rely exclusively on offsets and that critical data is aligned to optimize for local CPU caches, where possible.
4. Concurrency Model Overhaul: Enabling Concurrent Writes
The single greatest limitation of classic SQLite for modern applications is its pessimistic, file-level locking model. The requirement to acquire an $EXCLUSIVE$ lock for any write transaction dictates that concurrent writers are impossible and concurrent readers may sometimes be blocked, leading to performance degradation under heavy load. To scale across multi-core systems, VelociDB must adopt an internal concurrency model that allows high-volume reads to proceed without interruption and supports concurrent, non-blocking write attempts.
4.1. Moving Past File-Level Locking (Pessimistic Model)
By externalizing concurrency management to the operating system's VFS file locks, the original design sacrificed internal concurrency for robustness and simplicity. In a multi-core environment, this design is structurally limited. The new architecture transitions concurrency management internally, leveraging the fine-grained control afforded by the Rust language and modern database mechanisms.
4.2. Implementing Multi-Version Concurrency Control (MVCC)
Multi-Version Concurrency Control (MVCC) becomes the core mechanism for transactional isolation in VelociDB.
4.2.1. Snapshot Isolation
Under MVCC, every transaction receives a unique identifier or timestamp at its inception. Readers never block writers, and writers never block readers, because each transaction operates on its own consistent snapshot of the data, determined by its ID. When a write occurs, instead of overwriting the data in place, a new version of the record is created. Readers view the most recent committed version prior to their snapshot ID. This architectural choice eliminates read-write contention, maximizing the utilization of available CPU cores.
4.2.2. Version Chain Management and Storage Overhead
The benefit of non-blocking reads comes with a necessary complexity shift. Updates lead to version proliferation, resulting in increased disk usage and index size (storage overhead). Managing this storage requires maintaining version chains for records, complicating the B-Tree structure. The persistence layer must be engineered to handle this versioning efficiently, potentially utilizing a document-oriented approach where full records are rewritten and older versions are maintained for rollback or snapshot visibility.
4.2.3. Background Cleanup
To prevent indefinite inflation of the version store and performance degradation, a continuous, asynchronous background maintenance process is essential. This process is responsible for identifying and reclaiming space occupied by "dead" versions—those that are no longer referenced by any active transaction's snapshot. If transactions are long-running, they can stall this cleanup process, inflating the undo or version chains and slowing performance. The MVCC design therefore necessitates robust transaction duration management to ensure timely space reclamation.
4.3. Leveraging Multi-Core Processors Internally
MVCC handles concurrency at the data access level, but internal shared resources—such as the Pager's cache structure, thread pools, or asynchronous I/O queues—still present contention points, potentially negating the benefits of multi-core scaling.
4.3.1. Lock-Free Data Structures for Contention
To allow internal components to scale linearly with the number of processing cores, internal shared resources must be protected using non-blocking, lock-free data structures. These structures achieve thread safety without using mutual exclusion mechanisms (mutexes or semaphores). Instead, they rely on atomic instructions, such as Compare-and-Swap (CAS) loops, to manage state transitions. Examples include lock-free queues, ring buffers, and specific lock-free algorithms for managing tree structures. The shift to lock-free design minimizes the expensive overhead of operating system context switches and kernel involvement associated with mutex contention, allowing internal pathways to execute at high speeds and low latencies. This is critical for systems built on byte-addressable PMEM, where minimizing latency is the primary objective.
4.3.2. Optimistic Concurrency Control (OCC) for Metadata
For specialized, low-contention internal metadata (e.g., updating configuration settings, cache status flags, or the global transaction ID counter), Optimistic Concurrency Control (OCC) is adopted. OCC minimizes overhead by allowing transactions to proceed without acquiring locks, recording the state of the data before modification. The transaction validates its consistency at commit time—typically by checking if a version identifier (like an ETAG or sequence number) associated with the metadata has changed. If validation succeeds, the change is committed. If it fails, the change is transparently rolled back and retried. This approach yields high throughput when contention is rare, further optimizing core utilization by avoiding costly locking mechanisms.
5. Query Execution and Data Structure Optimization (The VDBE and B-Tree Layer)
The increased throughput delivered by high-speed I/O and MVCC means that the CPU and the VDBE (Virtual DataBase Engine) become the next architectural bottleneck. VelociDB must implement modern techniques to maximize CPU performance, specifically leveraging the cache hierarchy and vector processing capabilities.
5.1. Cache-Conscious Design for B-Trees and Data Layout
The B-tree remains the fundamental indexing structure due to its effectiveness in managing large datasets stored on persistent media and its logarithmic time complexity $O(\log N)$ for operations. However, the design of the B-tree nodes must be optimized for the CPU cache hierarchy, not solely for disk page size.
5.1.1. B-Tree Node Structure
Modern CPU systems rely heavily on cache coherence, and reading from main memory due to a cache miss incurs a significant time cost.34 B-tree nodes in VelociDB must be designed to align with CPU cache lines (typically 64 bytes). This involves careful field reordering and data alignment within the node structure to maximize the utility of data loaded into the cache (cache block utilization). The goal is to ensure that traversing the B-tree hierarchy results in as few cache misses as possible, leveraging the benefit B-trees already provide in reducing I/O operations by minimizing tree height.
5.1.2. Implicit Static Layouts
For indices that are primarily read-intensive, advanced layouts such as implicit static B-trees can be utilized. These structures eliminate explicit pointers between nodes, storing data in contiguous memory arrays using a specific layout (similar to Eytzinger). This dense, contiguous storage significantly improves data locality and reduces memory bandwidth usage during index scans, achieving substantial speed improvements by making the structure more CPU cache-friendly.
5.2. Vectorized Query Execution (SIMD)
The VDBE, which executes compiled SQL bytecode 2, must be transformed from a scalar processing unit to a vectorized engine to fully utilize modern CPU capabilities.
5.2.1. VDBE Vectorization
Vectorization, also known as data parallelization, converts algorithms from processing single operands (scalar) to processing multiple operands simultaneously (vector). Instead of fetching and processing one row or tuple at a time, the VDBE should operate on vectors (batches) of values (e.g., 256 or 1024 elements). This allows the execution time for a batch of operations to be nearly identical to the time required for a single scalar operation.
5.2.2. SIMD Application
Vectorized execution leverages Single Instruction, Multiple Data (SIMD) instructions (like AVX or SSE) inherent in modern CPUs. The VDBE must be specifically optimized for SIMD in computationally intensive stages, including: selection scans (applying WHERE clause filters), computing expressions, and performing aggregate functions (e.g., SUM, AVG). Furthermore, search operations within the cache-conscious B-tree nodes can be accelerated by using SIMD instructions to quickly find the correct position of a key within the node’s sorted array. Vectorization provides orders of magnitude performance increase, particularly for analytical workloads embedded within the application.
5.3. Hybrid Storage Layouts
Classic SQLite employs a row-based storage format, optimal for transactional systems that frequently access entire records (OLTP).41 Modern embedded applications, however, often require fast analytical capabilities (OLAP), such as generating statistics or dashboards on subsets of data columns.
To accommodate this mixed workload, VelociDB must adopt a hybrid storage strategy. While the primary data store maintains a row-based structure for transactional integrity, the VDBE must incorporate mechanisms—potentially specialized secondary indexes or in-memory vector buffers—to efficiently transform and present required column data in a columnar format during analytical queries.41 Columnar representation excels in analytical tasks because it allows high compression ratios (since data within a column is uniform) and reduces I/O by only reading the necessary columns.42 The columnar vector resulting from this transformation is the ideal format for rapid processing by the vectorized VDBE using SIMD.
6. The Internet Layer: Distributed Embedded Synchronization
The original SQLite was designed as a purely local file, with connectivity only addressed via the underlying OS file system. A modern embedded database must acknowledge the pervasive connectivity of edge devices and mobile platforms, supporting robust synchronization and cloud integration. The architecture must treat the local database as a replica that can operate fully offline.
6.1. Cloud-Based VFS Integration
6.1.1. Remote Storage VFS
The pluggable nature of the VFS layer provides the mechanism for integrating network storage. VelociDB would include a specialized Cloud VFS component designed to communicate with remote object stores (e.g., Amazon S3, Azure Blob, GCS). This VFS abstracts the cloud object store, treating the database file as a large, block-addressable object that is transparently accessed over the network.
6.1.2. Smart Partial File Access
To minimize network latency and bandwidth costs, the Cloud VFS must not download the entire database file. Instead, it must utilize network range-reads to fetch only the specific fixed-size pages (e.g., 4096 bytes) requested by the Pager layer. This approach is crucial for performance, especially when tasks only require accessing a small subset of a massive database file. The VFS layer must incorporate aggressive, block-level caching of recently accessed pages to avoid repetitive network calls, allowing client software to communicate efficiently with the object store and treating remote data as a locally cached block device.
6.2. Bi-Directional Synchronization and Consistency
The challenge of modern embedded data is bi-directional synchronization (two-way sync) between the local edge replica and a central cloud store. Traditional synchronization methods struggle with concurrent, disconnected edits, often leading to complex, proprietary conflict resolution mechanisms.
6.2.1. CRDTs as the Core Sync Protocol
Conflict-Free Replicated Data Types (CRDTs) provide the necessary mathematical rigor to solve the synchronization problem.48 CRDTs are data types designed such that concurrent modifications, performed on different replicas without coordination, can be merged deterministically without requiring complex, external conflict resolution logic.49 CRDTs guarantee strong eventual consistency, making them ideal for mobile apps, edge devices, and collaborative software where users frequently make offline edits.
6.2.2. Operation-Based Synchronization
The system must shift its synchronization unit from the entire database file state to an operational log. This log records atomic, CRDT-compliant update operations (CmRDTs or CvRDTs). By syncing this stream of operations—rather than file-level diffs—the system ensures that when two replicas exchange logs, the operations can be replayed and merged reliably, regardless of the order they arrive, guaranteeing convergence to the same final state.
6.2.3. Customizable Conflict Resolution
While CRDTs handle data merging inherently, real-world data often involves semantic conflicts (e.g., two users modifying a budget that depends on external application logic). The synchronization layer, built atop the CRDT framework, must allow for customizable conflict resolution policies. This involves tracking provenance and timestamps, enabling mechanisms like "last-write-wins" or application-defined logic to intelligently resolve conflicts that fall outside the scope of the base CRDT structures, ensuring data integrity across the distributed system.
7. Conclusion: The Modern SQLite (VelociDB) Blueprint
The architectural blueprint for VelociDB represents a comprehensive divergence from the foundational assumptions of its C-based predecessor, necessitated by the evolution of commodity hardware. The analysis confirms that the simplicity of the original file-based design is incompatible with the performance and concurrency demands of 21st-century systems.
The new architecture redefines the embedded database as a highly parallel, multi-core optimized, and asynchronously managed persistent data replica. The adoption of Rust provides the necessary guarantees for memory and thread safety, enabling the confident implementation of complex internal mechanisms like MVCC and lock-free concurrency control. This internal complexity, hidden behind the simple, zero-configuration interface, delivers exponential gains in performance by allowing concurrent reads and writes, maximizing core utilization, and exploiting SIMD instructions through a vectorized execution engine.
Furthermore, the integration of asynchronous, kernel-bypass I/O (DAX, io_uring) with PMEM-optimized logging drastically reduces transactional latency. The Internet layer, built around a Cloud VFS and utilizing CRDTs for consistency, transforms the local database from a static file into a resilient, bi-directionally synchronized component of a distributed system.
The resulting VelociDB retains the core promise of robustness and zero administration while transforming its internal operation to fully utilize modern advancements in processing, storage, and networking technologies. The implementation roadmap must prioritize stabilizing the Rust/FFI layer, developing the Async VFS/PMEM integration, and finalizing the MVCC/Lock-Free core before focusing on the Vectorized VDBE and CRDT synchronization integration.
Works cited
Why Is SQLite Coded In C, accessed November 11, 2025, https://sqlite.org/whyc.html
Deep Dive into SQLite's Internal Architecture - DEV Community, accessed November 11, 2025, https://dev.to/lovestaco/deep-dive-into-sqlites-internal-architecture-2fjl
File Locking And Concurrency In SQLite Version 3, accessed November 11, 2025, https://sqlite.org/lockingv3.html
What Modern NVMe Storage Can Do, And How To Exploit It: High-Performance I/O for High-Performance Storage Engines - VLDB Endowment, accessed November 11, 2025, https://www.vldb.org/pvldb/vol16/p2090-haas.pdf
SQLite concurrency and why you should care about it - Jellyfin, accessed November 11, 2025, https://jellyfin.org/posts/SQLite-locking/
Why Is SQLite Coded In C : r/programming - Reddit, accessed November 11, 2025, https://www.reddit.com/r/programming/comments/84fzoc/why_is_sqlite_coded_in_c/
Rust vs. C++: a Modern Take on Performance and Safety - The New Stack, accessed November 11, 2025, https://thenewstack.io/rust-vs-c-a-modern-take-on-performance-and-safety/
Rust vs Go in 2025 - Bitfield Consulting, accessed November 11, 2025, https://bitfieldconsulting.com/posts/rust-vs-go
Memory Safety in Rust | The AdaCore Blog, accessed November 11, 2025, https://blog.adacore.com/memory-safety-in-rust
How does Rust Achieve Complete Memory Safety as Opposed to C++? - Reddit, accessed November 11, 2025, https://www.reddit.com/r/rust/comments/131knig/how_does_rust_achieve_complete_memory_safety_as/
FFI - The Rustonomicon - Rust Documentation, accessed November 11, 2025, https://doc.rust-lang.org/nomicon/ffi.html
Tokio - An asynchronous Rust runtime, accessed November 11, 2025, https://tokio.rs/
Embedded devices - Rust Programming Language, accessed November 11, 2025, https://rust-lang.org/what/embedded/
SAN Basics: The Transition to NVMe | NetApp, accessed November 11, 2025, https://www.netapp.com/media/16913-san-basics-transition-to-nvme.pdf
Asynchronous I/O - IBM, accessed November 11, 2025, https://www.ibm.com/docs/en/informix-servers/12.10.0?topic=processors-asynchronous-io
Serverless Runtime / Database Co-Design with Asynchronous I/O - Apollo, accessed November 11, 2025, https://www.repository.cam.ac.uk/items/8a1cb0b8-c771-4325-a207-01821fd7e886
Non-volatile Memory Databases, accessed November 11, 2025, https://db.cs.cmu.edu/projects/nvm/
NVM: Is it Not Very Meaningful for Databases? - VLDB Endowment, accessed November 11, 2025, https://www.vldb.org/pvldb/vol16/p2444-koutsoukos.pdf
About Using a DAX-Enabled File System for Persistent Memory Database, accessed November 11, 2025, https://docs.oracle.com/en/database/oracle/oracle-database/21/ladbi/about-dax.html
Creating a DAX-Enabled File System for Persistent Memory Database - Oracle Help Center, accessed November 11, 2025, https://docs.oracle.com/en/database/oracle/oracle-database/26/ladbi/creating-dax.html
Introduction to pmemstream - PMem.io, accessed November 11, 2025, https://pmem.io/blog/2022/01/introduction-to-pmemstream/
Configure persistent memory (PMEM) for SQL Server on Windows - Microsoft Learn, accessed November 11, 2025, https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/configure-persistent-memory?view=sql-server-ver17
Memory-mapped files: pros and cons? - Stack Overflow, accessed November 11, 2025, https://stackoverflow.com/questions/8526498/memory-mapped-files-pros-and-cons
Memory-mapped files for efficient data processing | Oxford Protein Informatics Group, accessed November 11, 2025, https://www.blopig.com/blog/2024/08/memory-mapped-files-for-efficient-data-processing/
Mastering Multi-Core Processors in Embedded Applications - RunTime Recruitment, accessed November 11, 2025, https://runtimerec.com/mastering-multi-core-processors-in-embedded-applications/
Multiversion concurrency control - Wikipedia, accessed November 11, 2025, https://en.wikipedia.org/wiki/Multiversion_concurrency_control
Multiversion Concurrency Control (MVCC): A Practical Deep Dive - CelerData, accessed November 11, 2025, https://celerdata.com/glossary/multiversion-concurrency-control
Efficient Lock-Free Data Structure Protection | ArangoDB Blog, accessed November 11, 2025, https://arangodb.com/2015/08/lockfree-protection-of-data-structures-that-are-frequently-read/
DNedic/lockfree: A collection of lock-free data structures written in standard C++11 - GitHub, accessed November 11, 2025, https://github.com/DNedic/lockfree
Fear and Loathing in Lock-Free Programming | by Tyler Neely | Medium, accessed November 11, 2025, https://medium.com/@tylerneely/fear-and-loathing-in-lock-free-programming-7158b1cdd50c
Optimistic concurrency control - Wikipedia, accessed November 11, 2025, https://en.wikipedia.org/wiki/Optimistic_concurrency_control
Using Optimistic Concurrency Control With Duality Views - Oracle Help Center, accessed November 11, 2025, https://docs.oracle.com/en/database/oracle/oracle-database/26/jsnvu/using-optimistic-concurrency-control-duality-views.html
System Design — 8 Data Structures That Power Your Databases | by Neha Das - Medium, accessed November 11, 2025, https://medium.com/@nehud29/system-design-8-data-structures-that-power-your-databases-5ad119fbe63d
B-tree - Wikipedia, accessed November 11, 2025, https://en.wikipedia.org/wiki/B-tree
Cache-Conscious Data Structures - Microsoft, accessed November 11, 2025, https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/ccds.pdf
Implicit Static B-trees - Algorithmica, accessed November 11, 2025, https://algorithmica.org/en/b-tree
Ask HN: Books on designing disk-optimized data structures? | Hacker News, accessed November 11, 2025, https://news.ycombinator.com/item?id=32965075
15-721 Advanced Database Systems (Spring 2024) - 06 Vectorized Query Execution, accessed November 11, 2025, https://15721.courses.cs.cmu.edu/spring2024/notes/06-vectorization.pdf
Teaching Postgres New Tricks: SIMD Vectorization for Faster Analytical Queries | Tiger Data, accessed November 11, 2025, https://www.tigerdata.com/blog/teaching-postgres-new-tricks-simd-vectorization-for-faster-analytical-queries
Adapting Tree Structures for Processing with SIMD Instructions - OpenProceedings.org, accessed November 11, 2025, https://openproceedings.org/2014/conf/edbt/ZeuchFH14.pdf
Columnar vs. Row-Based Storage: Boosting Data Warehouse Speed, accessed November 11, 2025, https://www.dasca.org/world-of-data-science/article/columnar-vs-row-based-storage-boosting-data-warehouse-speed
Columnar database vs row database - Fivetran, accessed November 11, 2025, https://www.fivetran.com/learn/columnar-database-vs-row-database
The SQLite OS Interface or "VFS", accessed November 11, 2025, https://sqlite.org/vfs.html
C2FO/vfs: Pluggable, extensible virtual file system for Go - GitHub, accessed November 11, 2025, https://github.com/C2FO/vfs
Deadline Cloud virtual file system - AWS Documentation, accessed November 11, 2025, https://docs.aws.amazon.com/deadline-cloud/latest/userguide/storage-virtual.html
What Is Data Synchronization? Purpose, Types, Methods & Essential Tools | Estuary, accessed November 11, 2025, https://estuary.dev/blog/data-synchronization/
What is a bidirectional sync? Here's what you should know - Merge.dev, accessed November 11, 2025, https://www.merge.dev/blog/bidirectional-synchronization
About CRDTs • Conflict-free Replicated Data Types, accessed November 11, 2025, https://crdt.tech/
Conflict-free replicated data type - Wikipedia, accessed November 11, 2025, https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type
Real-Time Bi-Directional Database Sync for Operations - Stacksync, accessed November 11, 2025, https://www.stacksync.com/blog/real-time-bi-directional-database-sync-operations