Memtables, SSTables & Compaction

In this chapter

We'll trace what a write and a read actually do inside a Cassandra node — commit log, memtable, SSTables, and compaction — plus bloom filters, tombstones, and repair, all honestly explained as real internal mechanics this in-memory simulator can describe but not physically demonstrate.

9–11 min

The Problem in Real Life

Every INSERT GreenMart has run so far has felt instant and simple — a truck's location goes in, and it's just there. Mike asks the question that instant feeling hides: "where does it actually go? Is it just... in memory somewhere?"

Sarah realizes she's been treating every write as a single step, when a real Cassandra node treats it as several — and understanding those steps is the difference between trusting the fleet's data and just hoping it's fine.

If the power went out right now, would we lose that last truck ping or not?

Mike

One Step, Assumed vs. Several Steps, Understood

A write hits the commit log and a memtable

Durability on disk immediately, plus an in-memory sorted structure for fast recent access — both in the same step.

Memtables flush to immutable SSTables

Once full, a memtable is written to disk as an SSTable — never edited again, only ever superseded.

Compaction merges and cleans up

A background process merges SSTables over time, discarding anything overwritten or deleted — off the critical path of any write.

Bloom filters make reads cheap

A compact per-SSTable structure that can say "definitely not here" almost instantly, letting a node skip most files on a read.

Every write GreenMart runs — like the INSERT below — genuinely does two things at once on a real Cassandra node, not one. It's appended to a commit log on disk immediately, purely for durability (if the node crashes a second later, the commit log lets it recover). At the same time, it's written into a memtable — an in-memory, sorted structure holding recent writes for fast access, the actual reason reads come back so quickly right after a write.

This runs instantly in the simulator, and it would on a real node too — but for a different reason underneath. On real Cassandra, this INSERT hits the commit log and the memtable in the same step; the SELECT right after reads it straight out of that still-in-memory memtable, before anything's even touched a permanent data file.

The Same Write, Now Traced Through What Really Happens

CREATE TABLE truck_events (
  truck_id text,
  event_time text,
  speed_kmph int,
  PRIMARY KEY (truck_id, event_time)
);

INSERT INTO truck_events (truck_id, event_time, speed_kmph) VALUES ('truck-01', '2026-09-02T09:10:00', 47);

SELECT * FROM truck_events WHERE truck_id = 'truck-01';

This simulator keeps everything in memory permanently, with no commit log or on-disk files to actually demonstrate — the trace below is what a real node does, not something this Playground can show happening.

A memtable doesn't stay in memory forever. Once it fills up, Cassandra flushes it to disk as an SSTable ("Sorted String Table") — an immutable file, written once and never edited again. Over time, a busy table accumulates many SSTables, and a background process called compaction merges them, combining old and new versions of the same row into one, and permanently discarding anything that's been overwritten or deleted. This is exactly why Cassandra writes are so cheap: a write never has to find and edit an existing record in place — it just gets appended, and compaction sorts out the mess later, off the critical path of any single write.

Reading is the harder side of this trade. A read might need to check the memtable and several SSTables to find the current value of one row — so Cassandra uses a bloom filter per SSTable, a compact structure that can say "this SSTable definitely does not contain this row" almost instantly, letting a node skip most SSTables without ever opening them. A tombstone is what a DELETE actually writes — not an instant erasure, but a marker meaning "ignore any older value for this row," which only gets permanently cleaned up once compaction runs, the same honest "deletion isn't always instant" lesson Act 4's DynamoDB TTL chapter already taught in a different product. Finally, repair is the process that fixes replicas that fell out of sync — say, one node missed a write while it was down — comparing data across replicas and reconciling the differences, restoring the promise that every replica eventually holds the same data.

Key Takeaway

A write is cheap because it's only ever appended, never edited in place — a commit log for durability, a memtable for speed, and SSTables plus compaction to make that sustainable forever. A read pays the cost that design defers: checking multiple places, with bloom filters there specifically to skip the ones that don't matter.

Why This Matters

This explains why Cassandra can sustain the fleet's nonstop write volume without slowing down over time — writes never get more expensive as data grows, because nothing is ever edited in place. It also sets up the failure-handling chapter ahead: repair and hinted handoff both depend on understanding that a replica can genuinely fall behind, and needs a real mechanism to catch back up.

GreenMart now knows what's really happening every time a truck's location gets written and read back. None of this explains what happens when a node actually goes down mid-write, though — that's exactly where the next chapter goes.