Cheat Sheet

Where Does the Data Go? A Deep Dive Into the Modern Data Storage Landscape

8 core ideas from the full deep dive, condensed onto one page you can scan in two minutes.

#01

Theory

Theoretical Foundations

—ACID trades speed for guaranteed correctness; BASE trades correctness for speed and scale.
—CAP: during a network partition, pick Consistency or Availability — never both.
—PACELC extends this to normal operation too: Latency vs. Consistency.

Spanner

PC / EC

Cassandra

PA / EL

#02

Hardware

Physics & Hardware Latency

—CPU L1 cache ≈ 1 ns. RAM ≈ 100 ns. NVMe SSD ≈ 100 µs.
—RAM is roughly 1,000× faster than SSD, and 100,000× faster than a network call.
—Every caching decision is downstream of this one physical hierarchy.

RAM vs SSD

1,000×

RAM vs S3

100,000×

#03

Engines

Storage Engine Internals

—B+ Tree: in-place 8–16KB pages — fast, read-optimized (Postgres, MySQL, SQLite).
—LSM Tree: append-only MemTable + SSTables — write-optimized (Cassandra, RocksDB).
—WAL logs every mutation before it's applied, so a crash never loses a commit.

Read-heavy

B+ Tree

Write-heavy

LSM Tree

#04

Concurrency

MVCC & Isolation Levels

—MVCC: each write creates a new tuple version (xmin/xmax) instead of overwriting in place — reads never block writes.
—A background VACUUM process reclaims old tuple versions once they're no longer needed.
—Four isolation levels, loosest to strictest: Read Uncommitted, Read Committed, Repeatable Read, Serializable.

#05

Frontend

Client-Side Browser Storage

—LocalStorage: ~5MB, synchronous, XSS-readable — never store tokens there.
—Cookies: ~4KB, sent on every request — use HttpOnly + SameSite=Strict.
—OPFS + WASM-SQLite runs a real database inside the browser, off the main thread.

Cookie limit

4 KB

OPFS

Native file I/O

#06

Databases

Server-Side Databases

—Relational (Postgres): ACID, MVCC, B-Tree/GIN indexes, JSONB when you need flexibility.
—Wide-column (Cassandra/DynamoDB): masterless rings, tunable consistency via W+R>N.
—Redis serves sub-millisecond reads; Elasticsearch scores relevance with BM25.

Redis

< 1 ms

Elasticsearch

~10–50 ms

#07

Cloud

Cloud & Object Storage

—Object storage (S3/GCS): flat key-namespace, near-infinite scale, HTTP REST access.
—Block storage (EBS): a raw virtual disk, attached to exactly one instance.
—Ingress is free; egress isn't — moving data out is where cloud bills spike.

Erasure 8+4

50% overhead

Glacier restore

12–48 hrs

#08

Decisions

Architectural Scenarios

—Payments: Postgres (strict isolation) + Redis idempotency keys + an append-only audit log.
—Social feed: Cassandra (write-heavy) + Redis-backed pre-computed feeds.
—AI RAG: Postgres for metadata, pgvector/Qdrant for embeddings, S3 for raw documents.

Payments

ACID + WAL

AI RAG

pgvector + S3