Fan-out on Write vs Fan-out on Read
System designsystem design / scalability
3:02 AM, and the feed is dead
Your social app has 400k users. Home feed loads in 40ms. Then a footballer with 2.1M followers signs up, posts once — and every write starts timing out. Not reads. Writes. Posting a photo now takes 90 seconds.
Nothing about your read path changed. So why did a write break?
Because you chose fan-out on WRITE
Two ways to build a feed:
Fan-out on read (pull): store the post once. At read time, fetch the people I follow and merge their recent posts.
SELECT * FROM posts WHERE author_id IN (...300 ids...)
ORDER BY created_at DESC LIMIT 50;Fan-out on write (push): when someone posts, copy a row into every follower's inbox table. Reading is then one query on one index.
for follower in followers(author): # 2.1 MILLION times
insert(feed[follower], post_id)Push makes reads O(1) and writes O(followers). That's the trade you signed for — you just never met a user with 2.1M followers before.
The numbers that decide it
Say 1 post is read by 300 people on average.
- Pull: 1 write, 300 expensive merges.
- Push: 300 cheap writes, 300 trivial reads.
Feeds are read-heavy (often 100:1), so push usually wins. Until the follower count is extreme — then one write becomes millions of rows, and your queue backs up behind a single celebrity.
What real systems actually do: hybrid
This is the part people miss in interviews.
- Normal users → push. Fan out at write time into follower inboxes.
- Accounts above a threshold (say >100k followers) → do NOT fan out. Store once.
- At read time: read my precomputed inbox, then merge in the handful of "celebrity" accounts I follow, live.
You get O(1) reads for 99.9% of the timeline and skip the write explosion. Twitter published exactly this design; the threshold is a tunable number, not a law.
Three gotchas
- Deletes and edits. Push means the post is duplicated in millions of inboxes. Deleting it is now a second fan-out job. Store post_id only, never the post body — then delete = one row in
posts, and reads hydrate from there (a missing post = skip it). - Inactive users. Don't fan out to accounts that haven't opened the app in 30 days. Backfill on login instead. Often cuts write volume by half.
- Fan-out must be async. Doing it inside the POST /posts request handler is what caused the 90-second write. Enqueue a job, return 201 immediately, let workers chunk it (e.g. 1000 inserts per batch).
The generalizable idea
"Fan-out on write" = precompute at write time. It's the same decision as a materialized view, a denormalized counter, or a cached aggregate: you're moving work from the read path to the write path. The question is never "which is better" — it's "what's my read:write ratio, and what's my worst-case fan-out?"
Both numbers matter. The average tells you which to pick; the tail tells you where it breaks.
Your exercise
Pick a threshold for your own app. Given:
- 500k users, average 200 followers
- 5% of posts come from accounts with >50k followers
- Write budget: 200ms p99 on POST /posts
Answer two things:
(a) With pure push and async workers, how many inbox rows does one celebrity post generate, and how long does that job run at 5k inserts/sec?
(b) At read time in a hybrid model, what extra query do you run — and what has to be true about a user's "celebrities I follow" list for that merge to stay fast?
Bonus: what breaks if a user follows 800 celebrities?