The Evolution of Storage Eras
RDBMS → NoSQL → Distributed SQL → Vector — why today's storage landscape has this many moving parts.
Overview
Today's storage landscape has a lot of moving parts — relational databases, NoSQL stores, distributed SQL, vector databases — and it can look like the industry just kept inventing new databases for no reason. It didn't. Each new era showed up to solve a specific, real limitation of the era before it. Knowing that history makes the current landscape make sense, instead of looking like an arbitrary pile of options.
Why It Exists
Understanding why each era exists tells you what problem it was actually built to solve, which is far more useful than memorizing a list of database names. It also explains something that surprises a lot of people: most real companies aren't running one era's technology — they're running three or four eras' worth of tools at the same time, on the same system, because each addition solved a real problem when it was adopted and nobody had a reason to rip out the layer underneath it.
Real World Example
Picture a company founded in the late 1990s. It started on a single Oracle database — the only real option at the time (Era 1). In the 2010s, it added MongoDB for a new mobile app that needed a flexible, fast-changing schema (Era 2). Around 2020, it adopted a distributed SQL database like CockroachDB for a service that needed to stay consistent across multiple regions without giving up transactions (Era 3). And recently, it bolted on a vector database to power AI-driven search (Era 4). Nobody planned this as one clean architecture — it's four eras of storage technology, layered on top of each other, and that's completely normal.
How It Works
Era 1 (1980s–2000s), Monolithic RDBMS: everything ran on one relational database, scaled by buying a bigger single server — strict ACID guarantees, but a hard ceiling once internet-scale traffic arrived. Era 2 (2000s–2010s), NoSQL Explosion: systems like MongoDB and Cassandra gave up strict ACID and rigid schemas in exchange for scaling horizontally across cheap commodity servers. Era 3 (2010s–present), Distributed SQL & Cloud-Native: systems like Google Spanner and CockroachDB brought ACID transactions back, but running across globally distributed nodes instead of one machine, with cloud platforms decoupling compute from storage entirely (AWS Aurora, Snowflake). Era 4 (2023–present), Vector & Real-Time: AI workloads need to store and search high-dimensional embeddings, which is what pushed vector databases (Pinecone, Qdrant) and real-time analytical engines (ClickHouse) into the mainstream.
Diagram
Four eras, each solving the previous era's biggest limitation
Era 1: Monolithic RDBMS
Strict ACID, vertical scaling only — Oracle, DB2, MySQL
Era 2: NoSQL Explosion
Horizontal scale, eventual consistency — MongoDB, Cassandra
Era 3: Distributed SQL & Cloud-Native
Global scale with ACID restored — Spanner, CockroachDB, Aurora
Era 4: Vector & Real-Time
AI embeddings, real-time analytics — Pinecone, Qdrant, ClickHouse
Most Real Systems
run pieces of all four eras at once — an accumulation, not a replacement timeline
Common Mistakes
Assuming "newer" automatically means "better"
Why: Each era traded something away to gain something else — Era 2 gave up ACID for scale, for example. A newer era isn't a strict upgrade; it's a different set of trade-offs that may not fit your actual workload.
Fix: Judge a technology by whether its trade-offs match your problem, not by which era it belongs to.
Not expecting to run multiple eras' technology at once
Why: Teams sometimes treat having "more than one kind of database" as a sign something went wrong architecturally, when in most real, mature systems it's the expected outcome — see Chapter 1 on polyglot persistence.
Fix: Plan for a multi-era stack from the start rather than treating it as technical debt to eliminate.
Adopting Era 4 tooling without the operational maturity to run it
Why: A 40-year-old RDBMS has decades of tooling, documented failure modes, and hiring pool behind it. A cutting-edge vector database may not — and that gap is a real operational risk, not just a feature checklist difference.
Fix: Weigh a newer technology's ecosystem maturity (docs, tooling, hiring, community track record) alongside its technical features before betting production on it.
Interview Questions
What came first, relational databases or NoSQL databases?
Relational databases. They were the dominant model from the 1980s through the 2000s (Era 1), before NoSQL databases emerged in the 2000s–2010s (Era 2) specifically to handle horizontal scaling needs that relational databases struggled with at the time.
Why did NoSQL databases give up strict ACID guarantees, and what were they trying to solve?
They were trying to solve horizontal scale — spreading data across many cheap, commodity servers instead of one large, expensive machine. Strict ACID transactions are hard to coordinate across many distributed nodes without a serious latency cost, so NoSQL systems relaxed to eventual consistency (the BASE model) in exchange for scaling out more easily.
A legacy system built entirely on Era 1 (RDBMS) technology needs to add real-time, AI-powered search. How would you introduce Era 4 tooling without a full rewrite?
Add the vector database as a new, narrowly-scoped service rather than migrating the existing system — sync the relevant data into it (e.g. via Change Data Capture from the RDBMS's write-ahead log) so the vector store stays a read-optimized index, not a new source of truth. This keeps the blast radius small: the legacy RDBMS keeps owning the data it already owns, and the new capability is additive rather than a rewrite.
Production Best Practices
Do
✓Match the era/technology to the problem you actually have today, not to what's currently trending.
✓Expect to run multiple eras' technologies side by side in a mature system — that's normal, not a smell.
✓Evaluate a newer technology's operational maturity (docs, tooling, community) before betting production workloads on it.
Don't
✗Don't rip out a working RDBMS just because a newer era's technology exists.
✗Don't adopt Era 4 (vector/real-time) tooling without a concrete use case actually driving the need for it.
✗Don't assume the timeline is finished — expect a future era to eventually show up and repeat this same pattern.
Comparison
| Period | Defining Trait | Representative Tech | |
|---|---|---|---|
| Era 1: Monolithic RDBMS | 1980s–2000s | Strict ACID, single-node vertical scaling | Oracle, DB2, MySQL |
| Era 2: NoSQL Explosion | 2000s–2010s | Horizontal scale, BASE, eventual consistency | MongoDB, Cassandra, Redis |
| Era 3: Distributed SQL & Cloud-Native | 2010s–present | Global scale with ACID restored, serverless | CockroachDB, Spanner, Aurora |
| Era 4: Vector & Real-Time Analytical | 2023–present | AI embeddings, real-time analytics | Pinecone, Qdrant, ClickHouse |