Blog

Cache Stampede: What It Is and How to Fix It

What is Cache Stampede?

Imagine your app stores some data in a cache — like a database query that takes 2 seconds to run. When that cached data expires, the next request fetches it fresh. That is fine.

But what happens when 1,000 requests hit your server at the same moment the cache expires?

All 1,000 see an empty cache. All 1,000 run the same slow query. Your database gets hit hard.

That is a cache stampede — also called a Dogpile Problem or "thundering herd" problem.

A cache stampede happens when many requests all try to rebuild the same cached value at the same time. This overloads your database.

Why Does It Happen?

The timeline looks like this:

  1. Your cache holds data for /api/popular-posts with a TTL of 60 seconds.
  2. At second 60, the cache expires.
  3. 500 users request /api/popular-posts at that same moment.
  4. All 500 see a cache miss and each runs the same slow database query.
  5. Your database now handles 500 identical queries instead of just 1.

This gets worse the more traffic you have. The more popular the endpoint, the bigger the stampede.

Setup

First, let us set up the database client and Redis client we will use in all examples.

db.ts
import { SQL } from "bun";
import { drizzle } from "drizzle-orm/bun-sql";
import * as schema from "./schema";

const client = new SQL({
  adapter: "postgres",
  url: process.env.DATABASE_URL!,
});

export const db = drizzle({ client, schema });
redis.ts
import { RedisClient } from "bun";

// RedisClient lets you configure the connection explicitly
export const redis = new RedisClient(process.env.REDIS_URL ?? "redis://localhost:6379", {
  autoReconnect: true,
  maxRetries: 10,
  connectionTimeout: 3_000,
});
schema.ts
import { integer, pgTable, text, timestamp, uuid } from "drizzle-orm/pg-core";

export const posts = pgTable("posts", {
  id: uuid("id").primaryKey().defaultRandom(),
  title: text("title").notNull(),
  views: integer("views").notNull().default(0),
  createdAt: timestamp("created_at")
    .$defaultFn(() => new Date())
    .notNull(),
});

The Problem Code

Here is a basic cache-aside pattern that has the stampede bug:

cache.ts
import { desc } from "drizzle-orm";
import { db } from "./db";
import { redis } from "./redis";
import { posts } from "./schema";

async function getPopularPosts() {
  const cached = await redis.get("popular-posts");

  if (cached) {
    return JSON.parse(cached);
  }

  // WARNING: Every request that misses the cache runs this slow query
  const result = await db.select().from(posts).orderBy(desc(posts.views)).limit(10);

  await redis.send("SETEX", ["popular-posts", "60", JSON.stringify(result)]);

  return result;
}

When the cache expires, hundreds of requests all pass the if (cached) check at the same time and all hit the database together.

Fix 1: Mutex Lock with Bun RedisClient

The simplest fix is a distributed lock. Only one request gets to rebuild the cache. All others wait and then read the fresh value.

mutex-cache.ts
import { desc } from "drizzle-orm";
import { randomUUID } from "node:crypto";
import { db } from "./db";
import { redis } from "./redis";
import { posts } from "./schema";

// Try to grab a Redis lock for a key
async function acquireLock(key: string, ttlMs: number = 5_000): Promise<string | null> {
  const token = randomUUID();
  // NX = only set if the key does not exist
  // PX = auto-delete after ttlMs milliseconds
  const result = await redis.send("SET", [`lock:${key}`, token, "NX", "PX", String(ttlMs)]);
  return result === "OK" ? token : null;
}

// Release the lock — only if we own it
async function releaseLock(key: string, token: string): Promise<void> {
  // Lua script ensures we only delete the lock if we are still the owner
  const script = `
    if redis.call("get", KEYS[1]) == ARGV[1] then
      return redis.call("del", KEYS[1])
    else
      return 0
    end
  `;
  await redis.send("EVAL", [script, "1", `lock:${key}`, token]);
}

// Wait for a given number of milliseconds
function sleep(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

async function getPopularPosts() {
  const cacheKey = "popular-posts";

  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  // 2. Try to grab the lock
  const token = await acquireLock(cacheKey, 10_000);

  if (token) {
    // We got the lock — rebuild the cache
    try {
      // Check again in case another request built it while we were waiting
      const fresh = await redis.get(cacheKey);
      if (fresh) return JSON.parse(fresh);

      const result = await db.select().from(posts).orderBy(desc(posts.views)).limit(10);

      await redis.send("SETEX", ["popular-posts", "60", JSON.stringify(result)]);

      return result;
    } finally {
      await releaseLock(cacheKey, token);
    }
  } else {
    // Another request has the lock — wait and retry
    for (let i = 0; i < 10; i++) {
      await sleep(200);
      const result = await redis.get(cacheKey);
      if (result) return JSON.parse(result);
    }

    // Fallback: run the query if the lock holder never finished
    return db.select().from(posts).orderBy(desc(posts.views)).limit(10);
  }
}

What happens here:

  • Request #1 gets the lock → rebuilds the cache → releases the lock.
  • Requests #2–1000 wait in the retry loop → read the cache once it is ready.
  • The database only gets 1 query instead of 1,000.

Fix 2: Probabilistic Early Expiration (XFetch)

Locks work, but waiting requests still feel slow. A smarter approach is XFetch — some requests refresh the cache before it expires, using probability.

The closer you are to expiration, the higher the chance of an early refresh. No waiting, no stampede.

xfetch-cache.ts
import { desc } from "drizzle-orm";
import { db } from "./db";
import { redis } from "./redis";
import { posts } from "./schema";

interface CacheEntry<T> {
  value: T;
  delta: number; // how long the last fetch took, in seconds
  expiresAt: number; // unix timestamp in seconds
}

async function xfetchGet<T>(
  key: string,
  ttl: number,
  recompute: () => Promise<T>,
  beta: number = 1,
): Promise<T> {
  const raw = await redis.get(key);

  if (raw) {
    const entry: CacheEntry<T> = JSON.parse(raw);
    const now = Math.floor(Date.now() / 1000);

    // Decide if we should refresh early
    const shouldRefresh = now - entry.delta * beta * Math.log(Math.random()) >= entry.expiresAt;

    if (!shouldRefresh) {
      return entry.value;
    }
  }

  // Cache miss or early refresh — fetch fresh data
  const start = Date.now();
  const value = await recompute();
  const delta = (Date.now() - start) / 1000;

  const expiresAt = Math.floor(Date.now() / 1000) + ttl;
  const entry: CacheEntry<T> = { value, delta, expiresAt };

  await redis.send("SETEX", [key, String(ttl), JSON.stringify(entry)]);

  return value;
}

// Usage
async function getPopularPosts() {
  return xfetchGet(
    "popular-posts",
    60, // 60 second TTL
    () => db.select().from(posts).orderBy(desc(posts.views)).limit(10),
  );
}

How it works: Math.log(Math.random()) gives a negative number. As the cache gets closer to expiry, the formula makes a refresh more and more likely. The beta value controls how early the refresh can happen. A higher beta means more aggressive early refreshes.

Fix 3: Stale While Revalidate

Another simple approach: always return the cached value right away, even if it is old. Then refresh the cache in the background.

stale-cache.ts
import { desc } from "drizzle-orm";
import { db } from "./db";
import { redis } from "./redis";
import { posts } from "./schema";

interface StaleEntry<T> {
  value: T;
  refreshAt: number; // when to start a background refresh
  expiresAt: number; // when the cache entry is fully gone
}

async function staleWhileRevalidate<T>(
  key: string,
  softTtl: number, // seconds until background refresh starts
  hardTtl: number, // seconds until the cache entry is deleted
  recompute: () => Promise<T>,
): Promise<T> {
  const raw = await redis.get(key);
  const now = Math.floor(Date.now() / 1000);

  if (raw) {
    const entry: StaleEntry<T> = JSON.parse(raw);

    // Past soft TTL — refresh in the background, return old value now
    if (now >= entry.refreshAt) {
      rebuildCache(key, softTtl, hardTtl, recompute).catch(console.error);
    }

    return entry.value;
  }

  // Nothing in cache — must fetch now
  return rebuildCache(key, softTtl, hardTtl, recompute);
}

async function rebuildCache<T>(
  key: string,
  softTtl: number,
  hardTtl: number,
  recompute: () => Promise<T>,
): Promise<T> {
  const now = Math.floor(Date.now() / 1000);
  const value = await recompute();

  const entry: StaleEntry<T> = {
    value,
    refreshAt: now + softTtl,
    expiresAt: now + hardTtl,
  };

  await redis.send("SETEX", [key, String(hardTtl), JSON.stringify(entry)]);
  return value;
}

// Usage
async function getPopularPosts() {
  return staleWhileRevalidate(
    "popular-posts",
    50, // start background refresh after 50s
    120, // delete cache after 120s
    () => db.select().from(posts).orderBy(desc(posts.views)).limit(10),
  );
}

The trade-off: Users always get a fast response. But for a short window after 50 seconds, they may see data that is slightly old. For most content — like blog posts, dashboards, or product listings — that is fine.

Fix 4: Request Coalescing (Single-Flight)

ဤပြဿနာကို ဖြေရှင်းရန် အသုံးများသော နည်းလမ်းတစ်ခုမှာ Request Coalescing (သို့မဟုတ် Single-Flight) ဖြစ်သည်။

This is the most direct fix. When many requests ask for the same key at the same time, only one of them actually runs the query. The others wait for that one result and share it.

No Redis lock needed. No retry loop. The deduplication happens inside your server process using a simple in-memory Map.

single-flight.ts
import { desc } from "drizzle-orm";
import { db } from "./db";
import { redis } from "./redis";
import { posts } from "./schema";

// Tracks in-progress fetches — key → running Promise
// တူညီသော key အတွက် တစ်ပြိုင်နက် request များကို ဤ Map ဖြင့် ထိန်းသိမ်းသည်
const inflight = new Map<string, Promise<unknown>>();

async function singleFlight<T>(key: string, fetch: () => Promise<T>): Promise<T> {
  // If a fetch is already running for this key, join it — do not start a new one
  // ဤ key အတွက် fetch တစ်ခု ရှိနေပြီဆိုလျှင် ထိုအဖြေကိုပဲ အတူ စောင့်ယူသည်
  const existing = inflight.get(key);
  if (existing) return existing as Promise<T>;

  // We are first — run the fetch and let others join
  const promise = fetch().finally(() => {
    inflight.delete(key);
  });

  inflight.set(key, promise);
  return promise;
}

async function getPopularPosts() {
  const cacheKey = "popular-posts";

  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  // 2. Use single-flight to run only one DB query, no matter how many requests arrive
  // တစ်ချိန်တည်း request ပေါင်းများစွာ ဝင်လာသော်လည်း DB query တစ်ခုသာ run ရမည်
  return singleFlight(cacheKey, async () => {
    // Double-check cache inside the flight — another request may have built it
    const fresh = await redis.get(cacheKey);
    if (fresh) return JSON.parse(fresh);

    const result = await db.select().from(posts).orderBy(desc(posts.views)).limit(10);

    await redis.send("SETEX", [cacheKey, "60", JSON.stringify(result)]);
    return result;
  });
}

What happens here:

  • 1,000 requests arrive at the same time with an empty cache.
  • The first request starts the fetch and saves the Promise in the inflight Map.
  • All other 999 requests find that same Promise and wait on it — no extra DB queries.
  • When the first request finishes, all 1,000 get the result at once.

Limit: This only works within a single server process. If you run multiple servers (horizontal scaling), use the Mutex Lock approach with Redis instead — since inflight is in-memory and not shared across machines.

Comparison

| Approach | Latency | Freshness | Complexity | Scales Horizontally | | ---------------------- | -------------- | -------------- | ---------- | ------------------- | | Mutex Lock | Adds wait time | Always fresh | Medium | Yes | | XFetch | No extra wait | Near-fresh | Medium | Yes | | Stale While Revalidate | No extra wait | Slightly stale | Low | Yes | | Request Coalescing | No extra wait | Always fresh | Low | Single process only |

Which One Should You Use?

  • Mutex Lock — when data must be fresh at all times. Good for prices, stock, or permissions.
  • XFetch — when near-fresh is good enough and you do not want any waiting.
  • Stale While Revalidate — when speed matters most and slightly old data is okay.
  • Request Coalescing — when you run a single server and want the simplest possible fix.

Why RedisClient Instead of the Default redis Import?

Bun ships two ways to use Redis:

option-a.ts
// Option A — default client, zero config
import { redis } from "bun";
option-b.ts
// Option B — explicit client you control
import { RedisClient } from "bun";
const redis = new RedisClient(process.env.REDIS_URL!);

Use RedisClient when you need to control the connection — for example, setting autoReconnect, maxRetries, or connectionTimeout. In production, you always want those settings explicitly defined.

Summary

Cache stampede is easy to miss in development. But it can crash your server when real traffic hits.

The fix is simple. Pick the right approach based on how fresh your data needs to be:

  • Data must be exact → Mutex Lock
  • Near-fresh is fine → XFetch
  • Speed first → Stale While Revalidate
  • Single server, simplest fix → Request Coalescing

A good cache protects your database. A broken one can take it down.