Chapter 12 — Casebook II: Caching

Stale, Leaking, Stampeding — the Bugs That Live Between Your Code and Your Users

Part 3: The Casebook. Prerequisites: Chapters 1–11, especially Chapter 5 (DevTools, service workers) and Chapter 6 (curl, Cache-Control, CDN cache status). Theory: SD QA Q17 — Why caching is hard, Q18 — Cache strategies, Q20 — Eviction policies.

How to Read This Chapter

A cache is a second copy of some data, kept closer to the reader so it is faster or cheaper to get. Every cache bug comes from that fact: there are now two copies, and they can disagree. The copy can be too old (stale), someone else's (a leak), missing at the worst moment (a stampede), or in a shape your code no longer understands.

Caches are also hard to see: a request can pass through ten, and most answer silently. So first, the map.

The cache layers a request can hit

In the order a browser request meets them:

The last row differs in kind: the database keeps its buffer cache coherent, so it can be cold (slow after a restart) but never wrong. Every layer above it can be wrong. And a read replica behaves like a cache when it lags the primary (PostgreSQL Ch 17a). "I wrote it, then read the old value" is often a replica.

The vocabulary

Reading cache headers

Chapter 6 lists the Cache-Control directives. Four more headers say what a cache did:

• Age: 5400 — seconds in a shared cache. A large Age after a deploy means a pre-deploy copy.
• ETag — a fingerprint. A cache with an expired copy sends If-None-Match; if nothing changed the server replies 304 Not Modified, no body. A 304 means the cache checked.
• Last-Modified — the date version. A response with it but no Cache-Control or Expires may be cached heuristically — caches may guess a lifetime. "We set no cache header" does not mean "not cached."
• Vary: Origin — request headers added to the key. Without it, requests needing different answers share one entry.

Case 1 — "I Updated It, but the Page Still Shows the Old Value"

Symptom. An admin changes a price from 10 to 12. The database says 12; the product page says 10 — for about an hour. It happens once a week, never when you try.

First five minutes. redis-cli GET product:42 returns price 10; redis-cli TTL product:42 returns 2950. With a one-hour TTL, the stale value was written ten minutes ago — after the update that deleted this key. Logs by correlation ID (Chapter 7) show a page request spanning the update.

Hypotheses.

Root cause. Cache-aside has a race between a reader that misses and a writer that invalidates:

Fix. Layer the defences: always set a TTL to bound the worst case; add a delayed second delete or versioned writes; for hot keys, a lease (the cache issues a token on a miss and rejects a fill whose token a later delete invalidated, as Facebook described for memcache). Without coordination — locks, leases or versions — there is no perfect fix. A cache is eventually consistent with its source; you choose how long "eventually" may be.

Prevention. Decide per kind of data how stale it may be; read a checkout price from the database.

What an interviewer is testing: whether you can draw the miss/update/delete/stale-set interleaving and explain why the TTL is the backstop, not the fix.

Case 2 — "We Deployed the Fix but Users Still See the Bug"

The rule for deployable files

Keep old hashed assets online too — open tabs need them (Chapter 11 Case 6).

What an interviewer is testing: whether you can name every layer holding a copy, prove which one answered with headers, and explain why HTML and hashed assets need opposite policies. See FE-05 for the full strategy and FE-09 for the service-worker variant.

Case 3 — "A User Saw Someone Else's Data"

This is a security incident before it is a bug: contain, preserve evidence, find the cause, then scope who was affected so those handling privacy and legal duties can act.

Three ways a cache leaks data

A CDN caching a response with Set-Cookie can hand one user's session cookie to others. Many CDNs refuse by default — check yours.

What an interviewer is testing: that you contain before you diagnose, keep evidence, scope the exposure, and know everything that changes the answer belongs in the key.

Case 4 — "Everything Fell Over Right After Redis Restarted"

Symptom. Redis was restarted for an upgrade at 03:00. Within seconds database CPU hit 100%, p99 went from 80 ms to 30 s, and the pool threw timeout exceeded when trying to connect (Chapter 9).

First five minutes. Graph hit ratio, database queries per second and connections side by side (Chapter 10). At 03:00 the hit ratio fell from 98% to near zero, queries rose fiftyfold, connections hit the pool maximum. pg_stat_activity shows the same two queries — "top products" and a per-request config lookup — running hundreds of times at once.

Hypotheses.

Root cause. The database was sized for the 2% of reads that miss. This Redis ran without persistence, so it restarted empty and sent the database 100%, concentrated on hot keys. When 2,000 requests a second miss one key, cache-aside makes each run the same expensive query. This is a cache stampede (thundering herd, dog-piling):

The feedback loop turns slow into down: each extra query slows the rest, so the key stays missing longer and more requests pile up. No restart is needed for it either: synchronised expiry (a deploy writes thousands of keys with the same TTL 3600; an hour later they expire together), or one hot key expiring on the same minute every hour.

Fix. Several defences together (SD QA Q19 goes deeper):

• Request coalescing (single-flight) — concurrent misses in one process share one load:

const inFlight = new Map(); // key -> Promise

async function getCached(key, load, ttl) {
  const hit = await redis.get(key);
  if (hit !== null) return JSON.parse(hit);
  if (inFlight.has(key)) return inFlight.get(key);
  const p = (async () => {
    try {
      const value = await load();
      const jitter = Math.floor(Math.random() * ttl * 0.1);
      await redis.set(key, JSON.stringify(value), 'EX', ttl + jitter);
      return value;
    } finally {
      inFlight.delete(key);
    }
  })();
  inFlight.set(key, p);
  return p;
}

With 30 instances, 2,000 concurrent queries become at most 30.

• A per-key lock or lease across instances — the first miss takes SET lock:top-products <token> NX PX 5000; others wait briefly and re-read, or serve stale. The expiry stops a crashed holder blocking the key (SD QA Q28).
• Stale-while-revalidate — store a "soft" expiry before the real TTL; after it, one request refreshes while everyone else gets the slightly old copy.
• TTL jitter — a random few percent on every TTL so keys written together do not expire together (the code above does this).
• Warm-up and limits — pre-load hot keys, restart cache nodes one at a time, and cap database concurrency so a cold cache means slow pages, not a dead database.

Prevention. Load-test with an empty cache; alert on hit ratio.

What an interviewer is testing: that you see the cache as load-bearing capacity and can match single-flight, locks, stale-while-revalidate and jitter to what each prevents.

Case 5 — "The Data Never Updates" (Next.js)

Symptom. An admin edits a product; the Next.js product page shows the old text for hours, or until the next deploy. In next dev it updates instantly.

A caution first. Next.js caching changed significantly between major versions: defaults, option names and revalidation APIs have all moved. Check your version in package.json and read that version's caching docs. The method below does not change.

First five minutes. Reproduce with next build && next start — next dev renders on every request and hides most caching. Then go outside-in.

Root cause (here). The page's data was fetched in a way that let Next.js prerender it at build; the route table marked it static. The save handler wrote to the database and nothing else, so the only refresh was the next deploy.

Fix. Invalidate after the write: revalidatePath('/products/' + slug), or tag the fetch (say products) and call revalidateTag('products') to refresh every page that used it. Or set a revalidation window, or make the route dynamic. Option names, defaults and these functions' signatures differ between versions — confirm in your docs.

Prevention. Test "edit, then view" against a production build. The Next.js book explains the rendering and caching model.

What an interviewer is testing: that you separate CDN, server and client caches, reproduce in production mode, and treat invalidation as part of every write.

Case 6 — "Keys Disappear at Random"

Symptom. Backend: "a key with a 24-hour TTL is gone in twenty minutes." Frontend: "users get logged out at random, mostly at peak." A worker sometimes logs OOM command not allowed when used memory > 'maxmemory'.

First five minutes. Ask Redis:

redis-cli INFO memory | grep -E 'used_memory_human|maxmemory_human|maxmemory_policy'
# used_memory_human:3.99G
# maxmemory_human:4.00G
# maxmemory_policy:allkeys-lru
redis-cli INFO stats | grep -E 'evicted_keys|expired_keys'
# evicted_keys:18233419
# expired_keys:902113

Far more keys were evicted (thrown out for room) than expired.

Hypotheses.

Root cause. With maxmemory set and allkeys-lru, Redis makes room by removing the least recently used keys among all keys. At peak the cache grows, and sessions of users idle for a few minutes are the least recently used. Redis evicts them, and those users are logged out.

The eviction policies

• Under volatile-*, keys without a TTL are never evicted. If none have one, Redis behaves like noeviction.
• Redis LRU and LFU are approximate (it samples a few keys), so a recently used key can go.
• Defaults vary: self-managed Redis defaults to noeviction with no memory limit on 64-bit systems; managed services often set their own. Check CONFIG GET maxmemory-policy and your version's docs.

Fix. Separate what you can lose from what you cannot. Sessions, queues, locks and rate-limit counters go in a noeviction instance with headroom and an alert before full; the cache gets its own instance with allkeys-lru or allkeys-lfu. Find what fills memory with redis-cli --bigkeys and MEMORY USAGE key.

The in-process version. A module-level Map with no size limit and no TTL never evicts — it is a memory leak ending in an out-of-memory kill, exit code 137 (Chapter 8). Bound every in-process cache with an LRU limit plus a TTL.

What an interviewer is testing: expiry versus eviction, what each policy does to keys with and without a TTL, and why a cache and a session store should not share one.

Case 7 — "The New Product Is 404 for Ten Minutes"

Symptom. A product link shared at launch shows "Product not found" to some people for ten minutes.

First five minutes. Origin returns 200; the CDN returns 404 with cf-cache-status: HIT, age: 412. And redis-cli GET product:slug:aurora-lamp returns "null" with a TTL of 188.

Hypotheses.

Root cause. Someone — a colleague, a link-preview bot — opened the link before publishing finished. That miss found nothing, and cache-aside stored null for ten minutes: negative caching. It is often deliberate, to protect the database from repeated lookups for missing things — but here it had the positive TTL and nothing cleared it on publish. The 404 page also went out with cacheable headers, and many CDNs cache some error statuses briefly even without headers.

The same shape elsewhere:

• DNS. Query a subdomain before its record exists and resolvers cache the NXDOMAIN answer for a time set by the zone's SOA record. Creating the record does not clear it.
• Redirects. Browsers may keep a mistaken 301 long after you remove it. Use 302/307 until sure.
• Errors. A cached 500 turns a blip into minutes of failure.

Fix. Negative TTLs of seconds, not minutes. On publish, delete the negative entry and purge the URL. Never cache 5xx, or only for a second or two. Send no-store or a short max-age on the 404 page.

What an interviewer is testing: that absence can be cached, so create paths must invalidate just as update paths do.

Case 8 — "TypeError Right After Deploy, Only for Some Requests"

Symptom. After a deploy, TypeError: Cannot read properties of undefined (reading 'city') at formatShipping (src/orders/format.ts:22) on about 30% of order pages. The same orders always fail. Rolling back fixes it.

First five minutes. Line 22 reads user.address.city (Chapter 2). For a failing user, redis-cli GET user:42 returns {"id":42,"name":"Asha","city":"Pune"} — no address object. The release moved city into a nested address.

Hypotheses.

Root cause. The key stayed the same; the shape of the value changed. Old entries survive the deploy for their TTL and new code misreads them. During a rolling deploy old instances keep writing the old shape too, and may crash on new entries. This is version skew (Chapter 11 Case 6) stored in the cache.

Fix. Version the key prefix when the shape changes — v2:user:42 — so new code never reads old entries, which expire on their own. Read tolerantly: validate on read (a schema library such as zod, or a stored schemaVersion) and treat a mismatch as a miss. Flush deliberately: a flush, like a new prefix, is a cold start (Case 4). localStorage written by last month's frontend has the same bug: store a version and migrate or discard.

What an interviewer is testing: that you treat cached values as a versioned serialised format, like an API.

Case 9 — "Works in Incognito"

That sentence is evidence. Incognito starts with an empty HTTP cache, no service worker, no storage, no cookies, and usually no extensions, so the bug lives in profile state. Bisect it (Chapter 1): "Disable cache" (HTTP cache); "Bypass for network" (service worker, Case 2); clear only storage (old-shaped data, Case 8); clear cookies (stale session or flag); disable extensions.

One culprit is cache-shaped: a CORS response cached without Vary: Origin. A page on app.example.com fetches https://assets.example.com/config.json; the browser caches it with Access-Control-Allow-Origin: https://app.example.com. A page on admin.example.com then gets that cached copy, with the wrong allowed origin, and the browser blocks it. Incognito has no copy, so it works. A CDN can do this to every user (Chapter 11 Case 7, Chapter 6). Fix: send Vary: Origin whenever the allowed origin depends on the request.

What an interviewer is testing: that "works in incognito" narrows the bug to browser state, bisected one piece at a time.

Debugging Any Cache — the Checklist

1. Is a cache involved? Look for age, cache-status headers, Size column markers, 304s, hit/miss logs. If you cannot observe it, add the log.
2. Which layer answered? Bypass outside-in: CDN, origin via --resolve, app with cache skipped, database.
3. What is the key? Print it. Are user, tenant, locale, origin and version in it?
4. When was it written, and what is its TTL? Written after the change: a race (Case 1). Before: a missed invalidation or a long TTL.
5. How is it invalidated? Name the mechanism — TTL, delete on write, purge on deploy, revalidateTag — and check that it ran.
6. What happens on a miss under load? If every instance missed this key at once, could the source survive?

The Pattern Behind Every Case

The Causal Chains

The Rule You Keep Forever

A cache is a second copy of the truth with its own lifetime. When data looks wrong, find which copy answered, the exact key it was stored under, when it was written and how it was meant to die — before changing code. And before adding any cache, decide whose data it holds, how stale it may be, and what happens when it is empty.

Vocabulary Map

Practice Problems

Problem 1 — Map the layers. A Next.js site behind a CDN, with a service worker, Redis and Postgres. A user's new profile picture "won't show." List every layer that could hold the old image or URL, and the check for each.

Problem 2 — The race. Write the four-step interleaving that leaves a stale value after a delete in cache-aside. Then explain why deleting the key before the database update does not fix it either.

Problem 3 — Choose the headers. Give the Cache-Control for: (a) /index.html; (b) /assets/app.3e9f1b.js; (c) GET /api/me; (d) a public product list that may be a minute stale but should never make users wait for a refresh.

Problem 4 — Survive the cold start. Hit ratio is 99%. Describe what happens if the cache is flushed at peak, and four changes that let the database survive, with what each prevents.

Problem 5 — Redis triage. INFO shows maxmemory_policy:volatile-lru, memory at the limit, evicted_keys:0, and OOM errors on writes. Explain exactly what is happening and give two different fixes.

What's Coming Next

Chapter 13 covers the slow, intermittent, cumulative failures — tail latency, 502 and 504 chains, retries that multiply load, double writes, and one slow dependency taking down its callers.