Relational Databases & Scaling
The standard toolkit for scaling a relational database further — connection pooling, read replicas, and sharding, roughly in that order.
Overview
Once a single relational database server can't handle the load alone, there's a standard toolkit for scaling it further before reaching for a fundamentally different storage model — connection pooling, read replicas, and sharding, roughly in order of increasing complexity.
Why It Exists
A relational database's vertical scaling — a bigger server — has a hard ceiling, and even before hitting it, simpler problems show up first, like too many client connections overwhelming the database's connection limit. This toolkit exists because most real scaling problems are solvable at the relational layer with these specific, well-understood techniques, well before a team actually needs to reach for a different storage model entirely.
Real World Example
A serverless app scales to thousands of concurrent function invocations, each trying to open its own database connection — the database's connection limit (often a few hundred) gets exhausted fast. A pooler like PgBouncer sits in front, multiplexing those thousands of logical connections down to a small pool of real ones. Meanwhile, a reporting dashboard's heavy read queries get routed to a read replica instead of the primary, and once even that isn't enough, the largest tables get sharded across multiple nodes using a tool like Citus or Vitess.
Example Data
Connection pooling in numbers — 1,000 serverless invocations, one small pool of real connections
| Layer | Connections |
|---|---|
| Serverless function invocations | 1,000 concurrent |
| Real connections to the database (via PgBouncer) | 20 |
The Three Techniques, in Order
Connection Pooling (PgBouncer / RDS Proxy)
Multiplexes many client connections down to a small pool of real database connections. This is almost always the first, cheapest fix — most connection-limit problems have nothing to do with data volume at all.
Read Replicas
Asynchronous copies of the primary database that offload read traffic. They introduce replication lag, since a replica can trail the primary by a small but real amount of time.
Sharding (Citus / Vitess)
Splits a database horizontally across multiple independent nodes by a shard key, once a single node's storage or write throughput genuinely can't keep up. The most operationally complex option, and the hardest to undo later.
Diagram
Try these in order — most load problems never actually need to reach sharding
Single database server
vertical scaling has a ceiling
Connection pooling
fixes connection-count problems
Read replicas
fixes read-heavy load
Sharding
fixes write/storage volume — last resort
Common Mistakes
Reaching for sharding before trying connection pooling and read replicas
Why: Sharding is by far the most operationally complex option — resharding later is hard, and cross-shard queries and transactions get genuinely complicated. Most load problems are actually solved by the simpler two steps first.
Fix: Exhaust connection pooling and read replicas before considering sharding — measure whether they're actually enough before adding sharding's complexity.
Routing 'read-your-own-write' queries to a replica without accounting for replication lag
Why: A user who just wrote data and immediately reads it back from a lagging replica might not see their own change yet, which reads as a confusing bug to them.
Fix: Route read-your-own-write queries to the primary, or to a replica confirmed to have caught up, rather than any replica indiscriminately.
Choosing a shard key that doesn't distribute load evenly
Why: A shard key that concentrates traffic unevenly — like sharding by signup date when most active traffic is on recent users — creates hot shards that defeat the entire purpose of sharding.
Fix: Choose a shard key based on the actual access pattern and traffic distribution, not whichever column is most convenient.
Interview Questions
What problem does a connection pooler like PgBouncer solve?
It multiplexes a large number of client connections down to a small pool of real database connections, preventing the database's connection limit from being exhausted — a very common problem under serverless or highly concurrent workloads.
What's the risk of routing all read traffic to replicas without any exceptions?
Read replicas trail the primary due to replication lag. A user who just wrote data and immediately reads it back from a replica might see stale data that doesn't reflect their own recent write, which looks like a bug even though the system is working as designed.
How would you choose a shard key for a multi-tenant SaaS application's largest table?
I'd look at the real access pattern first — most queries in a multi-tenant app filter by tenant, so sharding by tenant ID often keeps each tenant's data on one shard and avoids cross-shard queries for their normal usage. The risk to watch for is a small number of very large tenants creating hot shards, which might need a secondary strategy (like splitting an oversized tenant further) rather than assuming tenant ID alone is always sufficient.
Production Best Practices
Do
✓Try connection pooling and read replicas before considering sharding.
✓Route read-your-own-write queries to the primary or a confirmed-caught-up replica.
✓Choose a shard key based on real traffic distribution, not convenience.
Don't
✗Don't jump straight to sharding without measuring whether simpler techniques are enough.
✗Don't route every read to a replica without considering staleness for recency-sensitive reads.
✗Don't pick a shard key that concentrates load on a small number of shards.
Comparison
| Complexity | Solves | Main Risk | |
|---|---|---|---|
| Connection Pooling | Low | Too many client connections | None significant |
| Read Replicas | Medium | Read-heavy load | Replication lag / stale reads |
| Sharding | High | Write throughput / storage volume | Hot shards, hard to reshard later |