Q17 — Why is caching hard — it's not just "store it in memory"?

What makes caching seem simple?

The pitch is seductive: your database is slow, so store frequently-read data in Redis (or Memcached, or an in-process map), serve it from memory, and your app is 100x faster. Put a get before the database call, a set after it. Done.

This works — until it doesn't. Phil Karlton's famous line: "There are only two hard things in computer science: cache invalidation and naming things." Caching seems simple because the fast path is trivial. The difficulty is entirely in the edge cases, failure modes, and consistency questions — and in production, those cases happen constantly.

Why is cache invalidation the hard part?

Every time data changes in your database, some cached representation of that data becomes stale. The question is: which cached keys need to be invalidated, and how do you know?

In a trivial case: you cache user:42 and user 42 updates their profile. You invalidate user:42. Simple.

But reality is messier. Consider what you might cache for user 42:

user:42 — their profile
feed:user:42 — their content feed
followers:user:42 — their follower list
search:engineers:san-francisco — a search result page that includes user 42 because they're an engineer in SF
recommendations:user:99 — recommendations for user 99, which happen to include user 42

User 42 changes their job title. Which of these caches are now stale? All of them. How do you know to invalidate recommendations:user:99? You don't, unless you maintain a reverse index of "which cache keys depend on user 42's data" — and maintaining that index is itself a complex, failure-prone problem.

This is the fundamental tension: the richer your caching strategy, the harder invalidation becomes. Caching derived data (aggregates, search results, recommendations) is where the value is highest, and also where invalidation becomes nearly impossible to get perfectly right.

Why is "just use a short TTL" not the answer?

The standard escape hatch: don't try to invalidate precisely — just set a short TTL (say, 60 seconds) and accept that the cache might be stale for up to that window.

This works for some data (profile pictures, blog post view counts). It doesn't work when:

Correctness matters: A user changes their password and logs out all sessions. If the session cache has a 60-second TTL, that user's old session is valid for up to a minute after they've explicitly invalidated it. A security vulnerability created by caching convenience.

Users see their own writes: User updates their bio, refreshes the page, sees the old bio. From the user's perspective, their update didn't work. This is called "read-your-own-writes" consistency, and it's invisible to your monitoring but infuriating to users.

Downstream systems depend on fresh data: A payment service caches a user's subscription status. A user cancels their subscription. For up to 60 seconds, the cache says they're still subscribed. Depending on what that service does, this could mean charging a cancelled user or granting access after cancellation.

Short TTLs reduce the staleness window but don't eliminate it. Every second the cache is stale is a window for inconsistency.

Why is deciding what to cache hard?

Not everything deserves to be cached. Caching the wrong things creates problems without helping performance.

Granularity: Do you cache the whole user object? Or individual fields? Whole object is simpler but means a single field change invalidates a large cache entry. Individual fields is precise but creates cache coherence problems — reading 10 fields means 10 cache lookups, and you might get field values from different points in time.

Cache coherence across keys: You cache user:42:name and user:42:email separately. Another service updates the user's name and email in one database transaction. The cache might briefly show the old name with the new email — a combination that never existed in the database. For most apps this is fine; for some it's a bug.

Hot data vs cold data: Caching data that's only accessed once wastes memory, evicting actually hot data. Caching data that changes every write causes constant invalidation with no benefit — you're doing more work than if you hadn't cached at all.

Write-heavy vs read-heavy: Caching makes sense when reads far outnumber writes. A user's tweet is read thousands of times but written once — great candidate. A real-time leaderboard that updates every second and is read just as frequently — the cache barely helps and adds a consistency problem.

Why is a cold cache dangerous?

When your cache starts empty — after a deployment, after a Redis restart, after a cache flush — every request misses the cache and hits the database. If your system is sized assuming most traffic hits the cache, your database cannot handle 100% of the traffic directly.

This is called a cold start problem. The database collapses under load it was never designed to handle alone, serving nothing, which means the cache never gets populated, which means the database stays under full load. A vicious cycle.

Solutions include cache warming (pre-populating the cache before routing traffic) and graceful rollout (slowly shifting traffic while the cache warms organically). Both require operational planning that "add a Redis" doesn't cover.

Why does a cache failure break everything if you're not careful?

If every read goes through cache.get() and only falls back to the database on a miss, then Redis going down means every request becomes a cache miss. Suddenly the database receives 100% of traffic — the traffic it was sized to handle with 90% going to cache. The database falls over. Your app is down.

The cache has become a critical dependency, not an optimization. You've traded one single point of failure (the database under load) for another (the cache). If the cache is not designed for high availability (with replication and failover), you've made your system more fragile, not less.

The mental model

code

Why caching is easy:
  Fast path: cache hit → return value, cache miss → fetch DB → populate cache

Why caching is hard:
  ├── Invalidation: which keys go stale when data changes?
  │     Simple data → manageable
  │     Derived / aggregated data → extremely hard
  │
  ├── Short TTL as escape: reduces staleness window but doesn't fix it
  │     Security, read-your-own-writes, downstream correctness still break
  │
  ├── What to cache: granularity, coherence, hot vs cold, write ratio
  │
  ├── Cold start: empty cache = database gets full traffic it can't handle
  │
  └── Cache failure: if cache is a critical path dependency, its outage = your outage

The hard parts are not in the code — they're in the consistency guarantees you're implicitly making
and the operational scenarios you haven't planned for.

Caching is easy to add and hard to reason about correctly. Every caching layer is an implicit contract with users about how stale their data might be — and that contract has to be made explicit, scoped to each type of data, and honored even when things go wrong.