Choosing a Storage System
56 chapters of trade-offs, distilled into the four questions that actually decide which storage system belongs in your architecture.
Overview
Every architecture in this journey — the payment system, the social feed, the e-commerce platform, the RAG application, the IoT pipeline — made its storage choices by answering the same small set of questions. This closing chapter pulls those questions out on their own, as a reusable framework, alongside a single comparative matrix covering every major storage system this journey has covered.
Why It Exists
It's easy to pick a database because it's popular, or because it's what the last project used, and only discover the mismatch once a workload outgrows it. This chapter exists as the one-page version of this entire journey: a deliberately reusable framework — consistency requirement, access pattern, scale, and read/write ratio — for evaluating a storage decision on its actual merits, whether or not the specific system you end up choosing was covered by name anywhere in these 56 chapters.
Real World Example
Revisit the five architectures from this path: the payment system chose a relational database because its ledger needed ACID above all else. The social feed chose a wide-column store and Redis because it needed to survive extreme write throughput with eventual consistency. The e-commerce platform used three different systems for three different sub-problems within the same product. The RAG application needed a vector store specifically for similarity search, paired with a plain document store. The IoT pipeline needed a time-series database matched to its append-heavy, time-ordered write pattern. None of these were arbitrary choices — each followed directly from asking the same four questions this chapter lays out.
The Four Questions That Decide a Storage System
What's Your Consistency Requirement?
Does this data need ACID guarantees because being briefly wrong has real consequences (a payment ledger, inventory counts), or is eventual consistency (BASE) an acceptable trade-off because staleness is invisible or self-correcting (a feed, a search index)?
What's Your Access Pattern?
Is this simple key-based lookup (a cache), full-text or faceted search (an inverted index), similarity search over high-dimensional data (a vector database), or time-ordered append-heavy writes (a time-series database)? The access pattern usually narrows the field more than any other factor.
What's Your Scale, and Does It Need to Be Distributed?
Does the data and its write volume genuinely exceed what a single well-provisioned node can handle, requiring sharding, replication, or a distributed SQL system — or is a single-node relational database, which is simpler to operate and reason about, actually sufficient?
What's Your Read/Write Ratio?
A read-heavy workload benefits enormously from caching and read replicas; a write-heavy workload needs a storage engine and data model built to sustain that write volume (see Storage Engines) rather than one optimized primarily for read performance.
Diagram
The framework, applied in order
1. Consistency requirement?
ACID vs. BASE
2. Access pattern?
key lookup, search, similarity, time-ordered
3. Scale — single node or distributed?
4. Read/write ratio?
cache-heavy vs. write-optimized
→ Storage system choice
Common Mistakes
Choosing a storage system because it's popular or familiar, without evaluating it against the actual workload's requirements
Why: A technology choice that worked well for a previous, differently-shaped workload can be a genuinely poor fit for a new one — popularity and familiarity aren't evidence of fit.
Fix: Run the workload through the four questions in this chapter explicitly, even for a technology the team already knows well.
Assuming one storage system must serve an entire application or product, rather than matching each sub-problem to its own best fit
Why: As the e-commerce chapter showed, a single product can have genuinely different consistency and access-pattern needs across its inventory, search, and cart — forcing one system to serve all of them usually means compromising on at least one.
Fix: Be willing to use polyglot persistence — multiple, specialized storage systems within one architecture — when the sub-problems genuinely differ.
Over-engineering for a distributed, eventually-consistent system when a single well-provisioned relational database would have been simpler and sufficient
Why: Distributed systems trade away simplicity and operational ease for scale and availability — paying that cost before the workload actually needs it adds complexity without a corresponding benefit.
Fix: Default to the simplest system that meets the actual current (and reasonably near-future) requirement, and revisit the decision if and when real scale demands it.
Interview Questions
What are the two most important questions to ask before choosing a storage system for a new feature?
What consistency guarantee does this data actually need (strong/ACID vs. eventual/BASE), and what's the primary access pattern (simple lookups, full-text search, similarity search, time-ordered writes)? These two questions eliminate most obviously wrong choices before considering anything else.
Why might a single product or application legitimately use several different storage systems at once, rather than standardizing on one?
Because different parts of the same product can have genuinely different requirements — as with the e-commerce platform's strongly consistent inventory versus its eventually-consistent search index versus its short-lived cached shopping cart. Forcing every part of the system into a single storage technology usually means at least one part is poorly served by a mismatched trade-off.
A team wants to adopt a distributed, eventually-consistent database for a new internal admin tool with a few hundred users. How would you evaluate whether that's the right call?
I'd walk through the same four questions: does this workload's consistency requirement actually need to trade away strong consistency (probably not, at a few hundred users, unless there's a specific reason), does the access pattern require anything a standard relational database can't handle well, does the actual scale exceed what a single well-provisioned node can serve (almost certainly not at that user count), and is the read/write ratio unusual enough to need special handling. For most internal tools at that scale, the answer to all four points toward a simple, single-node relational database being sufficient and considerably easier to operate — adopting a distributed system here would likely be paying for scale and availability guarantees the workload doesn't need, at a real cost in operational complexity.
Production Best Practices
Do
✓Run every new storage decision through the four questions in this chapter, even for familiar technology.
✓Use polyglot persistence when a product's sub-problems genuinely have different requirements.
✓Default to the simplest system that meets the actual current requirement.
Don't
✗Don't choose a storage system based on popularity or familiarity alone.
✗Don't force one storage system to serve fundamentally different sub-problems within the same product.
✗Don't adopt distributed-system complexity before the workload's actual scale requires it.
Comparison
| Best For | Consistency Model | Scales By | |
|---|---|---|---|
| Relational (Postgres) | Transactional data needing ACID | Strong | Vertical, then read replicas/sharding |
| Distributed SQL (Spanner/CockroachDB) | Global-scale transactional data | Strong, via consensus | Horizontal, built-in |
| Document/Wide-Column | High write throughput, flexible schema | Eventual (tunable) | Horizontal, native |
| Cache (Redis) | Low-latency key-based lookups | None needed — ephemeral | Vertical, then clustering |
| Search Index | Full-text and faceted search | Eventual | Horizontal (sharded indexes) |
| Vector Database | Similarity search over embeddings | Eventual | RAM/compression trade-offs (PQ, DiskANN) |
| Time-Series Database | Append-heavy, timestamped data | Eventual | Horizontal, time-partitioned |
| Object Storage | Large, infrequently-mutated blobs | Eventual (provider-dependent) | Effectively unlimited, horizontal |