Apache Cassandra
Wide-column storage at planetary scale: leaderless replication, tunable consistency.
Cassandra is a distributed wide-column store. A partition key scatters rows across a ring of equal nodes using consistent hashing; within a partition, clustering columns keep rows sorted. Writes go to any node, land in a commit log and memtable, and flush to SSTables — an LSM pipeline with leaderless replication and tunable read/write consistency.
Wide-column partitions
The partition key hashes to a ring position; inside the partition, clustering columns store rows sorted — billions of rows per partition with ordered scans.
Leaderless writes
Any node can coordinate a write; it goes to R replicas per the replication factor. Consistency is per-operation: ONE, QUORUM, or ALL — availability and latency against consistency.
Tombstones and compaction
Deletes write tombstones that shadow older values until compaction. Read path merges memtable + SSTables newest-first — the LSM read-amplification tradeoff.
Query-first schema design
Tables are designed around the queries: the partition key IS the access pattern. No joins, no ad-hoc secondary queries without explicit index tables.
The ring: consistent hashing without a leader
Every node owns a token range; each partition hashes to one. There is no master — a client can coordinate through any node. Replicas for a partition are the next N nodes clockwise on the ring; gossip keeps membership state.
The price of leaderless symmetry is consistency tuning: with QUORUM reads and writes you get R+W>N overlap; with ONE you get availability during partitions and eventual convergence.
CREATE TABLE sensor_readings (
sensor_id uuid,
ts timestamp,
value double,
PRIMARY KEY (sensor_id, ts)
) WITH CLUSTERING ORDER BY (ts DESC);
The storage pipeline is the LSM pipeline
Writes append to a commit log, mutate a memtable, flush into immutable SSTables, and compaction merges them — structurally identical to RocksDB. The walkthrough below traces exactly that loop; the Cassandra-specific parts are tombstone garbage collection and read-repair across replicas.
Schema design is query-first: one table per query pattern, denormalized on purpose. The wide-column shape lets each partition carry a different row shape — hence the family name.
Diagrams
static storyboard — WebGPU animates when available
A live WebGPU particle field tinted to this exhibit. Plays only with WebGPU and no reduced-motion preference; a static storyboard renders for everyone else.
[caption] Leaderless by design: the coordinator is whichever node received the write.
Any node coordinates; the write hits replicas per consistency level, appends to commit logs and memtables, and flushes to immutable SSTables that merge under compaction.
| puts | 2 |
| deletes | 1 (tombstone) |
| flushes | 2 |
| compaction dropped | c = 7 (shadowed) + expired tombstone |
| lookup(c) after compaction | 2 structures, not found |
All steps
- put(c = 5) — The write appends to the WAL (durability) and lands in the in-memory memtable. No disk read, no index update — this is why LSM writes are fast.
- put(c = 7) overwrites in memory — The overwrite coalesces in the memtable: the older c = 5 never reaches storage. One WAL record per put, though — the log keeps both.
- Flush → immutable run R3 — When the memtable fills it is flushed as a sorted, immutable run. Writes go to a new file; nothing in place is ever mutated.
- lookup(c) — memtable hit — The lookup checks the memtable first: c = 7 found after touching one structure. Recency is checked first because newer values shadow older ones.
- lookup(d) — miss, miss, hit — Memtable miss → R3 (newest run) miss → R2 hit, d = 4. Lookups may touch every level — the LSM read-amplification tradeoff.
- delete(c) — tombstone in memtable — A delete writes a tombstone marker, not an edit: LSM parts are immutable, so deletes must shadow older versions until compaction removes them.
- Flush → run R4 (tombstone) — The tombstone flushes into run R4. On disk now: R4 {c = ⊥}, R3 {c = 7}, R2, R1 — newest first.
- lookup(c) — tombstone wins — Memtable miss → R4 has the tombstone: stop. “Not found (deleted)” — the older c = 7 in R3 is shadowed and never read.
- Compact L0 + L1 — Compaction merges R1–R4 with L1 in one key-sorted pass: c = 3 and c = 7 are dropped (a newer tombstone shadows them), and the tombstone itself is dropped because no older run below can hold another c.
- Result — Lookup cost returns to 2 structures; disk holds one version per key. Compaction trades write amplification for read speed — the LSM bargain.
LSM write path, newest-first lookup, a delete via tombstone, and a compaction. Teaching model: RocksDB-flavored memtable + L0 runs + one merged L1 run; real engines compact by level with size ratios and keep tombstones until they cannot shadow anything older.
Generic LSM mechanics; Cassandra adds replica read-repair and per-SSTable bloom filters on top.
[caption] Generic LSM mechanics; Cassandra adds replica read-repair and per-SSTable bloom filters on top.
The write/read/delete/compaction loop Cassandra shares with every LSM engine — traced end to end on a small key family.
One sensor partition holding clustering-column rows — newest-first ordered scans over billions of points.
[caption] Illustrative partition row counts for one sensor at one-minute resolution over ~5 years.
One sensor partition holding clustering-column rows — newest-first ordered scans over billions of points.