Cache Stampede (Thundering Herd).

🎬The scenario

3:00 PM. Your homepage feed is cached in Redis with a 60s TTL. Traffic: 5,000 req/s, all served from cache in <1ms. CPU on the database is basically idle. Life is good.

3:01 PM. The key expires.

In the same millisecond, ~5,000 in-flight requests all check Redis, all get a MISS, and all decide "guess I'll rebuild it myself." Now 5,000 identical, expensive queries hit the database at once. It falls over. Requests pile up, TTLs on OTHER keys expire while everyone waits, and the misses cascade. Your cache didn't prevent the outage — its expiry TRIGGERED it.

That's a cache stampede (a.k.a. thundering herd, or dogpile).

🔍Why it bites intermediate folks

The naive read-through looks totally correct:

v = cache.get(key)
if v is None:
    v = db.expensive_query()   # <-- 5,000 threads run this at once
    cache.set(key, v, ttl=60)
return v

Nothing is "wrong" here. The bug only appears under concurrency + a hot key + a synchronized expiry. It's invisible in dev and in code review.

🛠️Three fixes, pick per situation

1. Locking / request coalescing — only ONE caller rebuilds; the rest wait for it or serve stale.

if cache.get(key) is None:
    if lock.acquire(key, nx=True, ex=30):   # only one winner
        v = db.expensive_query()
        cache.set(key, v, ttl=60)
        lock.release(key)
    else:
        return cache.get_stale(key)          # others reuse old value

2. Early / probabilistic recomputation — refresh BEFORE expiry so a real miss almost never happens. XFetch algorithm:

if now - delta*beta*ln(rand()) >= expiry:
    recompute()   # one lucky caller refreshes early, others keep hitting cache

3. TTL jitter — never expire a whole class of keys at the same instant.

ttl = 60 + random.randint(0, 15)   # spread the herd across time

⚠️The trap

Adding MORE cache layers or a shorter TTL feels protective but makes stampede WORSE — more frequent synchronized expiries, more simultaneous misses. The fix is never "expire more often"; it's "don't let everyone miss at the same time" + "don't let everyone rebuild at once."

🎯Your exercise

Take the naive read-through above. Add a per-key Redis lock (SET key val NX EX 30) so exactly one caller rebuilds and the others return the last-known value. Then answer: what happens if the ONE caller that holds the lock crashes mid-query — and how does the EX 30 expiry save you from a permanent deadlock?