Cache Stampede Prevention

The mechanism is always the same shape: a key that many concurrent requests need becomes a miss for all of them within the same narrow window, and each one independently does the expensive recompute — hitting the database, calling the origin API, running the aggregation — instead of one request doing the work and the rest reusing it. The multiplier is exactly the number of concurrent requests that missed in that window, which is precisely largest for your hottest keys, the ones a cache exists to protect.

Three situations produce this reliably. A hot key's TTL expires under sustained concurrent load — the most common case, and the one that scales with traffic, so it gets worse exactly when you can least afford it. A cold cache after a restart or deploy — every key is a miss simultaneously, and the database absorbs the full unfiltered traffic pattern the cache was hiding. A cache node failure or flush — the same cold-start effect, but unplanned and often during an incident that's already stressing the system.

The direct fix: when a request misses the cache, it first tries to acquire a short-lived lock for that key (SETNX in Redis, or an equivalent atomic "set if not exists"). Only the request that wins the lock actually queries the database and repopulates the cache; every other concurrent request either waits briefly and then reads the now-populated cache, or is told to retry shortly. The database sees one query per stampede window instead of one per concurrent request, regardless of how many thousands were waiting.

The lock needs its own short TTL as a safety net — if the lock holder crashes after acquiring it but before populating the cache, a lock with no expiry would leave every subsequent request waiting forever for a value that will never arrive. In-process, this is the same idea as Go's singleflight package: identical concurrent calls collapse into one in-flight execution that all callers share the result of, without even needing a distributed lock when all the traffic lands on one process.

Instead of every request treating a key as valid right up until the exact expiry instant then all missing together, probabilistic early expiration has each read compute a small, growing probability of proactively recomputing before expiry — weighted by how expensive the last recomputation was. A cheap-to-compute value might only trigger early refresh in its final second; an expensive aggregation might start probabilistically refreshing well before its TTL ends. Because the probability triggers independently per request, refreshes spread out across many small, low-probability events instead of colliding in one instant when the TTL hits zero.

The well-known form of this (sometimes called XFetch) uses the last recomputation's duration as the weight: now + delta * beta * ln(random()) compared against the stored expiry, where delta is how long the last recompute took and beta tunes how aggressively to front-load the risk. The practical effect: for a hot key, one request refreshes it slightly early, well before the herd would have hit a hard miss — most of the time, no request ever sees a stampede-triggering miss at all.

Stale-while-revalidate takes a different trade: instead of preventing the miss, it removes the cost of the miss from the client's critical path entirely. When a key expires, every concurrent request is served the stale (already-expired) value immediately — no one waits on the database — while exactly one background request refreshes the cache for the next round of readers. Requests that need strict freshness can't use this; requests that can tolerate a brief staleness window get both zero added latency and zero stampede.

HTTP has this built in: Cache-Control: max-age=60, stale-while-revalidate=30 tells a compliant cache to serve the cached response for up to 30 seconds past its 60-second freshness window while it revalidates in the background, and CDNs implement this natively. The same idea applies inside an application-level cache: keep the expired value around for a grace period, serve it while one request (using the same mutex-lock pattern above) refreshes it, and only fall through to a genuine blocking database call if the grace period has also elapsed.

Trigger. A flash-sale product page's cache entry, read by roughly 5,000 requests per second at peak, expires at exactly 2:00:00pm with no coalescing in place. Symptom. In the same second, the database sees a spike from its normal near-zero direct load for that row to 5,000 concurrent identical queries, exhausts its connection pool, and every other tenant sharing that database starts timing out too — a caching bug becomes a database-wide incident.

Root mechanism. The cache had no mechanism to ensure only one request repopulates an expired key — every miss independently treated itself as the first and only request needing to recompute. Recovery: add mutex-locked recomputation immediately (fastest to deploy, direct fix for the multiplier) and confirm via connection-pool metrics that concurrent queries for that row drop from thousands to one during the next expiry. Verification: replay the same traffic pattern against a staging environment with the lock in place and confirm the database sees exactly one query per expiry window regardless of concurrent request count. Prevention: layer in stale-while-revalidate for this specific hot key so future expirations never even produce a visible miss to the client, and add jittered TTLs across all product-page cache entries so the next flash sale's keys don't all expire in the same synchronized instant.