Designing a Social Media Feed
The opposite problem from a payment system: correctness matters less here than surviving a write volume that would crush a relational database.
Overview
A social media feed is close to the opposite design problem from the payment system in the previous chapter: instead of needing every write to be strongly consistent, it needs to survive an enormous, bursty write volume, and a feed that's a few seconds stale is completely unnoticeable to users.
Why It Exists
Every post, like, and comment is a write, and a single popular post can generate write volume that would overwhelm a traditional relational database's single-writer model within seconds. This chapter exists to walk through the core feed-architecture decision — fan-out on write versus fan-out on read — and why real systems end up using both, precisely because no single strategy handles both an average user and a celebrity account well.
Real World Example
A regular user with 200 followers posts a photo — fanning that post out by immediately writing it into all 200 followers' pre-computed feed lists is fast and cheap. A celebrity account with 50 million followers posts the same kind of photo — fanning it out to 50 million individual feed lists at once would be a massive write spike for one single post. Real systems handle this by fanning out on write for typical accounts, and falling back to fanning out on read (merging the celebrity's posts into a follower's feed at read time) specifically for accounts above a follower-count threshold.
Example Data
The same post, two very different fan-out costs
| Account Type | Followers | Fan-Out-on-Write Cost |
|---|---|---|
| Typical user | 200 | 200 feed-list writes |
| Celebrity account | 50,000,000 | 50 million feed-list writes — avoided via fan-out on read |
The Feed Architecture, Piece by Piece
Fan-Out on Write — Precompute Each Follower's Feed
When a post is created, it's immediately written into the pre-computed feed list of every follower, making reads fast (just fetch the precomputed list) at the cost of a write for every follower.
Fan-Out on Read — Merge Feeds at Request Time
Instead of precomputing, a follower's feed is assembled on demand by merging recent posts from everyone they follow — cheaper to write, but more expensive per read, since the merge happens every time the feed is requested.
The Hybrid Approach — Handling the Celebrity Problem
Real systems fan out on write for accounts under a follower threshold, and fan out on read (merging in at request time) specifically for accounts above it — getting fast reads for the common case without the celebrity write-storm.
Wide-Column Storage + Redis Caching
Posts themselves are stored in a wide-column store (see Document & Wide-Column Stores) built for high write throughput, while precomputed feed lists live in RedisA single-threaded, in-memory key-value data structure store, commonly used for caching, sessions, and rate limiting.Learn more for fast, low-latency reads.
Diagram
Two fan-out strategies, chosen per-account based on follower count
Typical account posts
fan out on write → all followers' feed lists
Celebrity account posts
no fan-out — merged in at read time instead
Follower requests feed
precomputed list + merged celebrity posts
Common Mistakes
Using fan-out on write unconditionally, regardless of follower count
Why: A single post from a high-follower-count account can generate a write spike large enough to degrade the entire system for every other user at the same time.
Fix: Use a hybrid strategy: fan out on write below a follower-count threshold, and fan out on read above it.
Choosing a strongly consistent relational database as the primary store for post and feed data at this scale
Why: A single-writer relational model isn't designed for the sustained, bursty write throughput a large-scale social feed generates, and the feed doesn't actually need the strong consistency guarantees a relational database provides.
Fix: Use a wide-column store built for high write throughput for posts, accepting eventual consistency as the appropriate trade-off for this workload.
Serving every feed read directly from the primary data store instead of a cache
Why: Feeds are read far more often than they're written — hitting the primary store on every read wastes its capacity on a workload a cache is much better suited for.
Fix: Cache precomputed feed lists in Redis, and only fall back to the primary store on a cache miss.
Interview Questions
What's the difference between fan-out on write and fan-out on read for a social feed?
Fan-out on write immediately copies a new post into every follower's precomputed feed list at post time, making reads fast but writes expensive for high-follower accounts. Fan-out on read instead assembles a feed on demand by merging posts from everyone a user follows at request time, making writes cheap but reads more expensive.
What is the 'celebrity problem' in feed architecture, and how is it usually solved?
It's the fact that a pure fan-out-on-write approach requires writing a new post into millions of feed lists at once for an account with millions of followers, creating a massive write spike. It's usually solved with a hybrid approach: fan out on write for typical accounts, and fall back to fan-out on read (merging that account's posts in at request time) specifically for accounts above a follower-count threshold.
Why is eventual consistency an acceptable trade-off for a social feed, when it wouldn't be for the payment system covered in the previous chapter?
The cost of being wrong differs enormously between the two systems. A feed showing a post a few seconds late, or a like count that's briefly stale, has essentially no real consequence for the user or the business. A payment system showing an incorrect balance, even briefly, has direct financial and legal consequences. Because the feed's failure mode is low-stakes and self-correcting (the data catches up shortly), it's worth trading strict consistency for the write throughput and availability eventual consistency provides — a trade-off that would be unacceptable for the ledger in the payment system.
Production Best Practices
Do
✓Use a hybrid fan-out strategy based on follower count, not one strategy universally.
✓Store posts in a wide-column store built for high write throughput.
✓Cache precomputed feed lists in Redis, and read from the primary store only on a cache miss.
Don't
✗Don't fan out on write unconditionally regardless of an account's follower count.
✗Don't use a strongly consistent relational database as the primary store at this write volume.
✗Don't serve every feed read directly from the primary data store.
Comparison
| Write Cost | Read Cost | Best For | |
|---|---|---|---|
| Fan-out on Write | High for high-follower accounts | Low — precomputed | Typical accounts, below a follower threshold |
| Fan-out on Read | Low — no precomputation | Higher — merged at request time | High-follower ('celebrity') accounts |