Thundering Herd & Cache Stampede

When a popular cache entry expires, every concurrent request may miss at once and hammer the database — the thundering herd problem. At scale this can take down your database in seconds. A related pattern, cache stampede, happens when many requests simultaneously try to regenerate the same expired entry. Solutions exist at every level. Locking (or mutex): only one request regenerates the value; others wait or get the stale value. Request coalescing (singleflight): duplicate in-flight requests for the same key share one backend call. Jittered TTLs: instead of every key expiring at exactly 300 seconds, expire between 270–330 seconds so expirations spread over time. Probabilistic early expiration: each request has a small chance of refreshing the cache before TTL expires, spreading regeneration load gradually.

// Cache expires at exactly 12:00:00
// 1000 requests arrive at 12:00:01
// All 1000 query the database simultaneously
async function getProduct(id) {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);

  // No lock — every miss hits DB
  const product = await db.query('SELECT ...', [id]);
  await redis.setex(`product:${id}`, 300, JSON.stringify(product));
  return product;
}

const inflight = new Map();

async function getProduct(id) {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);

  const key = `product:${id}`;
  if (inflight.has(key)) return inflight.get(key);

  const promise = db.query('SELECT ...', [id])
    .then(async (product) => {
      const ttl = 270 + Math.random() * 60; // jittered
      await redis.setex(key, ttl, JSON.stringify(product));
      inflight.delete(key);
      return product;
    });

  inflight.set(key, promise);
  return promise;
}