▣datahall
← back to gallery
embedded-sqlCest. 2000Public Domain

SQLite

The most-deployed database in the world: a single-file SQL engine with WAL, MVCC, and zero ops.

SQLite is a serverless, transactional, self-contained SQL engine that runs in-process and stores the entire database in a single cross-platform file. It implements most of SQL-92, supports B-tree and r-tree indexes, materialized views, JSON, full-text search, and ACID transactions across both rollback-journal and write-ahead-log journal modes.

Footprint
< 750 KiB
Journal
WAL or rollback
Indexes
B-tree + r-tree + FTS5
Concurrency
many readers / 1 writer

Single-file storage

One .sqlite file holds the schema, tables, indexes, and free pages. The file is portable across architectures and works read-only when needed.

Write-Ahead Log (WAL)

WAL appends uncommitted changes to a sidecar file so readers and writers never block each other. The WAL is checkpointed back into the main file periodically.

B-tree pages

Tables and indexes are stored as B-trees of fixed-size pages (default 4 KiB, configurable up to 64 KiB). Each page has a small header followed by cells.

In-process SQL compiler

A full SQL parser, planner, and bytecode VM lives in your process. There is no daemon, no socket, no authentication layer -- that is also why it is fast.

On-disk layout

A SQLite file begins with a 100-byte header that contains the magic string "SQLite format 3", the page size, the schema cookie, and pointers to the freelist and the most recently committed transaction. The rest of the file is a sequence of equally-sized pages.

Pages are typed: lock-byte, freelist, B-tree interior, B-tree leaf, payload overflow, and pointer-map. The B-tree interior pages form a navigational index; B-tree leaf pages hold actual row data. Pages that are too small to fit a row spill into a chain of overflow pages linked by 4-byte next-page pointers.

  • A 4 KiB page can hold 2-3 typical rows, or 40+ small rows.
  • Schema changes bump the schema cookie so prepared statements from older schemas are re-prepared automatically.
  • PRAGMA integrity_check walks every page to detect corruption.

Concurrency model

SQLite serializes writers and parallelizes readers. In rollback-journal mode, a writer takes an EXCLUSIVE lock, while readers use SHARED locks. In WAL mode, writers append to the WAL file and readers see the most recently committed snapshot plus the WAL tail, which lets readers and one writer run simultaneously without blocking.

WAL mode is the default for new databases that opt in via PRAGMA journal_mode = WAL. It is faster under mixed workloads and survives crashes because the WAL is replayed or rolled back at next open.

              PRAGMA journal_mode = WAL;
PRAGMA synchronous = NORMAL;
PRAGMA busy_timeout = 5000;
            

Query evaluation

SQLite uses a flat-bytecode VM. The query planner builds a tree of operations -- open-read, open-write, sort, filter, aggregate, join -- and the executor dispatches each op in turn. There is no JIT, no vectorized engine, no parallelism. For the workloads SQLite targets (embedded, low-concurrency, single-user) this trade-off wins: a 600 KB library that boots in microseconds and never pages out.

Diagrams

page-microscopeA 4 KiB SQLite page, under the microscopeclick nodes for description
sqlite page · 4 KiBpage header (8 B)page type · first freeblock · cell count · cell content areacell pointer arraygrows downunallocatedcell content areagrows uptuple detail (full mode)payload lenrowidrecord headerdataoverflow page chain

[caption] Real pages also include a freeblock chain. The pointer-map page tracks parent/child relationships and is used by autovacuum.

A single SQLite B-tree leaf page with header, cell-pointer array, and cell content area.

wal-timelineWrite-ahead log: append, commit, checkpoint, truncateclick nodes for description
timeappend framewrite to .wal sidecarcommitfsync the WALcheckpointmerge into main db filetruncatereset WAL headercrash / power losson restart: replay WAL → consistent state

[caption] In rollback-journal mode the durability step is replaced by an exclusive lock and a journal copy.

How WAL mode serializes writers and replays/rolls back at next open.

pipelineRead path: SQL -> parser -> bytecode VM -> pager -> B-tree leaf pageclick nodes for description
SQLexecutionparser, planner, bytecodeVMpagermemoryfetches pages from disk orWALB-treeindexwalks pages, finds rowB-tree leaf pagestorageparses cell, returns tuplemode: read

The end-to-end read path through SQLite, from SQL text to returned tuple.

pipelineWrite path: SQL -> WAL append -> B-tree update -> checkpointclick nodes for description
SQLexecutionparser, planner,bytecode VMWAL appenddurabilityframe written towrite-ahead logpagermemoryupdates B-tree pagesin memoryB-treeindexpages split or mergeB-tree leaf pagestoragenew tuple versionpersistedmode: write

The end-to-end write path through SQLite, with WAL durability and eventual checkpointing.

diagramOn-disk file layoutclick nodes for description
SQLite · On-disk file layout
file header100 Bmagicpage_sizeschema cookiefreelist headwal / journalpage 1B-tree interiorpage 2B-tree leafpage 3B-tree leaf…more pageswrite-ahead log (separate file, journal or wal mode)

100-byte header, then a chain of B-tree pages, with the WAL as a sidecar file.