Chapter 19: The System Design Framework — From Blank Problem to Justified Architecture

Curriculum: Distributed Systems from First Principles Chapter: 19 of 24 Part: 9 — Synthesis Prerequisites: All of Chapters 1–18. This chapter is the integration layer — it teaches you to wield the toolkit you've built. Estimated reading time: 80–90 minutes

Opening

Every chapter up to now has taught you one layer of the foundation: how disk works, how databases index data, how replication propagates writes, how consensus guarantees agreement, how caches absorb load, how message queues decouple systems, how load balancers distribute traffic, how observability makes systems knowable. You now have 18 tools in your toolbox.

The system design interview — and the real work of a senior engineer — is not a test of whether you know about consistent hashing. It's a test of whether you can start from a vague requirement ("design a system that handles X") and, through structured reasoning, arrive at an architecture where every component exists because of a specific constraint and every trade-off is consciously chosen. The failure mode of smart engineers is jumping to solutions ("we should use Kafka here") before they understand why Kafka is the right answer for this specific problem. The interviewer watches this happen and thinks: this person will over-engineer things.

This chapter gives you a framework — not a rote checklist but a thinking process — for decomposing any system design problem from first principles. Then it demonstrates the framework on a complete worked example, showing exactly how the analysis leads to architecture decisions. After reading this, you'll understand the shape of how engineers at Google, Meta, and Stripe actually think through system design — and you'll be able to replicate that thinking in any interview or real architecture session.

Part 1: Why Most People Fail System Design

The most common failure pattern in system design interviews: the candidate begins drawing boxes immediately. "We need an API server, a load balancer, a database, Redis for caching, Kafka for events..." Within 3 minutes they have a complex architecture diagram. The interviewer asks: "Why Kafka?" The candidate says: "For reliability, because it's distributed." The interviewer asks: "How many events per second?" Silence.

This candidate knows about Kafka — they've used it, maybe they read Chapter 15. But they don't know why this system specifically needs Kafka, because they haven't established the system's scale, access patterns, or consistency requirements. They're pattern-matching to architectures they've seen, not reasoning from the problem's constraints to a solution.

The second common failure: over-engineering the first version. The candidate designs a globally distributed, sharded, multi-region active-active system for a product that serves 1,000 users. Interviewers and real teams want to see that you start with the simplest thing that works and add complexity only where the constraints demand it. Complexity has a cost — ops burden, latency, failure modes. Unnecessary complexity is a design smell.

The third failure: not making trade-offs explicit. Every design choice involves trade-offs. Using write-through caching (Chapter 9) improves read performance but increases write latency. Using synchronous replication (Chapter 4) improves durability but increases write latency. Using a message queue (Chapter 14) decouples services but adds eventual consistency. Candidates who don't name these trade-offs appear to not know they exist — which is worse than having chosen differently.

The framework below addresses all three failure modes.

Part 2: The Framework — Six Steps

Step 1: Clarify Requirements (5–8 minutes)

Before drawing a single box, ask questions that define the problem space. Requirements fall into two categories:

Functional requirements: What does the system do? These define the features. Write them as verbs: "users can post tweets," "users can follow other users," "the feed shows posts from followed users in reverse chronological order."

Non-functional requirements: How well does the system do it? These define the constraints that drive architecture. The critical ones:

• Scale: How many users? How many requests per second? Peak vs. sustained?
• Read/write ratio: Is this a read-heavy system (Wikipedia: 99% reads) or write-heavy (logging: 99% writes)? Mixed (Twitter: ~10 reads per write)?
• Data size: How much data is stored? How much per user? Growth rate?
• Latency: What does the user experience? Read latency < 100ms? Write latency < 500ms? Is eventual consistency acceptable (news feed) or is strong consistency required (bank balance)?
• Availability: What's the SLO? 99.9%? 99.99%? (Chapter 18's SLOs apply immediately here)
• Durability: Can we lose data on hardware failure? (Affects replication strategy — Chapter 4, Chapter 5)
• Consistency: Does the user see their own write immediately (read-your-writes)? Can different users see different versions temporarily?

The clarifying questions you must ask:

• "How many users are we designing for — now and at peak?" (defines scale)
• "What's the read/write ratio of the core operations?" (defines storage and caching strategy)
• "Is strong consistency required or is eventual consistency acceptable?" (defines replication and consensus strategy)
• "What's the acceptable latency for reads and writes?" (defines caching and synchrony requirements)
• "What's the data retention requirement?" (defines storage cost)

You don't need to ask all of these — context determines which matter. For a URL shortener: consistency matters (the same short URL must always map to the same long URL), scale matters (billions of redirects), and write volume is low. For a real-time chat system: delivery latency matters more than strong consistency, and write volume is high. For a bank: strong consistency is non-negotiable, latency requirements are moderate.

Write the requirements down during the interview. This is not formalism — it's an anchor. Every subsequent design decision should trace back to a requirement. "We're using write-through cache here because the read/write ratio is 100:1 and we established that reads must be < 50ms." If you don't write it down, you'll drift and make inconsistent decisions.

Step 2: Estimate Scale (5 minutes)

Once you have the requirements, compute the numbers. This is the back-of-envelope calculation — rough arithmetic that establishes the order of magnitude of the problem. The numbers tell you which bottlenecks matter.

The Chapter 1 latency numbers are the anchors:

• Memory access: ~100ns
• SSD random read: ~100μs (1,000× slower than memory)
• Network round trip (same datacenter): ~500μs
• Disk read: ~1ms–10ms
• Network round trip (cross-datacenter): ~50ms–200ms

The estimation formulae you need:

Requests per second:

Daily active users (DAU) × actions per user per day / 86,400 seconds

Storage per year:

Writes per second × data size per write × seconds per year
= writes/s × bytes × 31,536,000

Bandwidth:

Requests per second × data size per request

Memory for cache (if caching hot data):

Total working set size × cache hit rate target
Working set = data accessed by 80% of requests (often 20% of total data — Pareto principle)

An example: "Design Twitter's feed"

Given:
  500M registered users, 100M DAU
  Users check feed ~5 times/day
  Users post ~2 tweets/day (DAU × fraction who post ≈ 50M posts/day)
  Average tweet: 300 bytes

Read load (feed checks):
  100M users × 5 checks/day / 86,400 ≈ 5,800 reads/second

Write load (new tweets):
  50M tweets/day / 86,400 ≈ 580 writes/second

Feed read:write ratio: 5,800 / 580 ≈ 10:1

Storage (tweets):
  50M tweets/day × 300 bytes = 15 GB/day = 5.5 TB/year

Media (photos, videos — separate):
  50M tweets/day × 20% with media × 500KB average = 5 TB/day (handled by CDN, not in scope)

Peak load (assume 3× average):
  Read peak: ~17,500/second
  Write peak: ~1,700/second

These numbers immediately tell you several things: reads dominate by 10:1, so caching is important. Write volume (580/s) is modest — a single PostgreSQL primary can handle 10,000–50,000 writes/second with appropriate indexing. The scale is large enough to require horizontal read scaling but not so extreme that writes need sharding immediately. 5.5 TB/year of tweet text is manageable in a single database for the first few years.

The purpose of estimation is not to get the exact number — it's to answer: what's the hardest part of this problem? For Twitter's feed, the hard part is not writing tweets (moderate volume), it's reading the feed efficiently for 100M users.

Do the five-minute estimate live. Guess first, then reveal the arithmetic — and notice which number tells you the hard part of the problem.

Step 3: Define Core Entities and Data Model (5 minutes)

Before choosing technology, define what data the system stores. Draw the core entities and their relationships. This is effectively an ER diagram at a high level.

For Twitter:

The data model reveals access patterns. The core read operation is: "given a user ID, return their feed — the 20 most recent tweets from users they follow, sorted by created_at descending." This query traverses:

1. Follow table to get the list of user IDs this user follows (could be 0–5,000 followees)
2. Tweet table to get recent tweets from all those user IDs
3. Sort and paginate

This is a complex, expensive query on a normalized relational schema. The challenge is step 2: "get recent tweets from N user IDs" — this is an IN clause with potentially thousands of IDs, and you need to sort across all their tweets by time. On a single database this works at small scale. At 5,800 reads/second with users following thousands of accounts, it doesn't.

The data model is where you identify the fundamental tension in the system. For Twitter: normalised data storage is efficient for writes (one tweet written once) but expensive for feed reads. Denormalised data storage (pre-computing each user's feed) is expensive for writes but cheap for feed reads. This tension will drive the biggest architectural decision.

Step 4: Design the Core Flows — Read Path and Write Path Separately

This is the heart of system design. Design the read path and write path independently, then integrate. Different constraints govern each.

Read path design: Start from what the user sees and work backward. The user requests their feed. Where does the feed come from? A database query? A pre-computed feed in a cache? What's the latency requirement? What's the cache hit rate?

Write path design: Start from the write event and trace forward. A user posts a tweet. What systems are updated? The tweet is stored. Followers' feeds might be updated (if pre-computing). Notifications might be sent. Analytics might be updated. What happens synchronously (must complete before the write is acknowledged) and what happens asynchronously (can happen after the response is returned)?

For Twitter's feed, two fundamental architectures:

Fan-out on read (pull model):

Writes are cheap: O(1) — just store the tweet. Reads are expensive: O(F) where F = number of followees. For a user following 1,000 people, a feed read is a query across 1,000 user IDs. At 5,800 feed reads/second with users following 500 accounts on average: the Tweet table gets 5,800 × 500 = 2.9M lookups/second. That's heavy.

Fan-out on write (push model):

Reads are cheap: O(1) — just read the pre-computed feed. Writes are expensive: O(F) where F = number of followers. For a celebrity with 10M followers, one tweet triggers 10M feed list updates. At 580 tweets/second average, the write amplification is 580 × avg_followers. If average followees count is 200, that's 580 × 200 = 116,000 feed updates/second — manageable for a message queue (Chapter 14) to process asynchronously. But the celebrity problem remains: one tweet from a user with 10M followers triggers 10M writes in a burst.

Hybrid (what Twitter actually uses):

The hybrid handles the celebrity problem by not pre-computing celebrity tweets — instead, they're fetched at read time (there are few celebrities to check, so this is O(num_celebrities_followed) ≈ O(5) for most users). This is the actual architectural decision Twitter made, documented publicly. The insight comes from the data model analysis in Step 3.

Step 5: Select Storage and Scale

Now, with the access patterns established from Step 4, choose the storage technologies. The selection criteria are the access patterns — not your familiarity with a technology.

The storage decision tree:

For Twitter: tweets → PostgreSQL (write once, read by ID or time range). Feed lists → Redis sorted sets (pre-computed, sorted by timestamp, O(1) reads). Media → S3 + CDN. Follow graph → PostgreSQL (adjacency list: follower_id, followee_id indexed both ways). User search → Elasticsearch.

Scaling the read path:

Once you've chosen the storage technology, add the scaling layers:

1. Read replicas (Chapter 4, 5): Distribute read load across multiple PostgreSQL replicas. 10 replicas handle 10× the read load of a single primary. Add before adding caches — replicas are simpler and add durability.

2. Caching (Chapter 9): For hot data accessed repeatedly (user profiles, recent tweets), add Redis in front of the database. Calculate the cache hit rate: if 20% of tweets account for 80% of reads (Pareto), caching 20% of the tweet data in Redis absorbs 80% of the database read load. Choose the eviction policy (LRU for recency-biased access like feeds) and TTL.

3. Sharding (Chapter 7): When reads and writes both exceed a single machine's capacity. Shard by user_id for user data — all data for a user lives on one shard, enabling efficient per-user queries. Shard tweets by tweet_id for the tweet table, but shard the feed lists by user_id (the user's feed is always read/written as a unit).

Scaling the write path:

1. Asynchronous processing with message queues (Chapter 14, 15): Fan-out (copying a tweet to 10M follower feeds) must be async. The write API acknowledges as soon as the tweet is stored; a queue (Kafka topic) carries the fan-out work to background workers. The user doesn't wait for 10M cache updates before getting a response.

2. Write-ahead log for replication (Chapter 5): All writes go to the primary PostgreSQL. The WAL streams to replicas. Not a scaling decision — it's a reliability decision. Covered in requirements clarification (durability requirement).

3. Rate limiting writes (Chapter 17): Protect the system from write amplification spikes (celebrity tweets) by rate-limiting the fan-out workers. They process the queue at a sustainable rate, not at burst speed that might overwhelm Redis.

Step 6: Address Failure Modes and Trade-Offs Explicitly

Every design has failure modes. Name them and say what you've done about them:

Single points of failure: Every component you added — what happens when it fails? Redis failing while the cache is unavailable: reads fall back to the database (load increases, but the system stays up — this is a degraded mode, not a full outage). Kafka failing: new tweets are stored but fan-out stops — feeds go stale but are still readable from cache. Name these degraded modes.

The consistency trade-off for feeds: With fan-out on write, a user's feed is pre-computed in Redis. After they post a tweet, their own feed doesn't immediately include their own tweet (the fan-out is async). This violates read-your-writes consistency. Fix: when a user loads their feed, check if their own recent tweets (from the Tweet table directly) are in it; inject them if not. Or: use synchronous fan-out for the user's own feed, async for all other followers.

The hot partition problem: A celebrity tweet triggers massive fan-out. Rate-limit the fan-out as above, and use the hybrid model (don't fan-out celebrities). But also: what if a non-celebrity goes viral? Their follower count can jump from 10,000 to 1M in hours if a tweet is retweeted by a celebrity. Monitor follower_count as a metric; when it crosses the threshold, migrate that user from fan-out mode to hybrid mode asynchronously.

The backfill problem: A new user follows 500 existing users. Their feed cache is empty. The first request must be served by the fan-out-on-read path (querying the Tweet table for all 500 followees' recent tweets). This is expensive but rare (only on first follow or after long absence). Cache the result and serve subsequent requests from cache.

Part 3: Communication — How to Talk Through a Design

System design is partly a communication skill. You're not just producing an architecture — you're demonstrating how you think. The interviewer cares about the reasoning, not just the output.

Narrate your decisions: Don't silently draw boxes. Say what you're doing and why. "I'm choosing to put the feed in Redis sorted sets rather than querying the Tweet table on every read because we established that reads are 10× writes, and the feed query requires joining across potentially thousands of user IDs — that's expensive at 5,800 reads/second." This lets the interviewer follow your reasoning and challenge it productively.

Propose before committing: Before committing to a component, mention alternatives. "I could handle fan-out synchronously in the write path, but that would add latency to tweet posting proportional to the number of followers. For a celebrity with 10M followers, that's untenable. So I'll use a message queue to do it asynchronously." This shows you know the trade-off space.

Ask for feedback at decision points: "Does this seem like the right trade-off given the constraints, or should I approach the fan-out differently?" Interviewers often have a specific constraint in mind they haven't mentioned. Inviting input doesn't show weakness — it shows collaborative design skills, which is what senior engineers actually do.

Don't aim for perfection — aim for completeness: A design that covers all the major components with honest trade-offs is better than a design that is perfect in one area and missing others. Cover: data model, read path, write path, storage selection, scaling approach, and at least two failure modes. If you spend 30 minutes perfecting the database schema and never discuss caching or the fan-out problem, you've failed to demonstrate breadth.

The time allocation for a 45-minute system design interview:

0–8 min:  Clarify requirements and write them down
8–13 min: Estimate scale (the key numbers)
13–18 min: Data model and core entities
18–33 min: Design core read and write paths
33–42 min: Deep dive into the hardest component (guided by interviewer)
42–45 min: Failure modes and trade-offs

Part 4: The Storage Selection Cheat Sheet

This summary connects Chapter 2–10 concepts to storage decisions:

Relational database (PostgreSQL, MySQL): Use when: data has relationships that require joins. Strong consistency required. Queries are varied and hard to predict in advance (OLTP). Less than ~10TB of hot data. Writes < 50,000/second.

Avoid when: write volume exceeds single-primary capacity. Schema is highly variable. Data is naturally key-value (overhead of relational features isn't used). Time-series data (wrong data structure — Chapter 10's LSM trees are better).

Redis: Use when: data fits in memory. Access pattern is key-value or simple data structures (sorted sets, lists, sets, hashes). Caching hot database results. Real-time counters, rate limiting state (Chapter 17). Session storage.

Avoid when: data doesn't fit in memory. Need complex queries. Persistence/durability is paramount (Redis persistence is weaker than PostgreSQL — Chapter 2's WAL is more robust).

Cassandra / DynamoDB: Use when: write volume exceeds single-machine capacity (Chapter 10's LSM trees handle high write throughput). Data is key-value or wide-row (read/write by partition key). Geographic distribution required. Access patterns are known and simple — no ad-hoc joins.

Avoid when: complex queries, ad-hoc aggregations, or strong consistency required. Chapter 6's consistency model: Cassandra defaults to eventual consistency — appropriate for some use cases, wrong for others.

Kafka (message queue): Use when: decoupling producers and consumers. Fan-out to multiple consumers. Async processing of writes. Event sourcing (the log is the truth). Data retention for replay.

Avoid when: immediate consistency required. Message count is small (SQS or a database queue is simpler). Consumers need complex query patterns on the queue (Kafka is a log, not a queryable store).

Elasticsearch: Use when: full-text search. Complex aggregations across large datasets. Fuzzy matching, ranking. Log search (Chapter 18: ELK stack).

Avoid when: primary data store (use as a secondary index on top of PostgreSQL or Cassandra). Strong consistency required (Elasticsearch is eventually consistent).

S3 / Object storage: Use when: storing large binary blobs (images, videos, audio, backups). Unlimited capacity. Cheap at scale. CDN integration.

Avoid when: data needs to be queried (S3 is not a database — it's a key-value store for objects). Latency must be sub-millisecond (S3 first-byte latency is ~10ms).

Part 5: The Worked Example — Design a Pastebin / Code Sharing Service

Let's apply the full framework to a concrete problem different from Twitter, demonstrating that the framework is general, not Twitter-specific.

Problem statement: "Design a service like Pastebin — users can create a text snippet (up to 10MB), get a short URL, share it, and anyone with the URL can view the snippet. Snippets optionally expire."

Step 1: Clarify Requirements

Functional:

• Create a snippet (text, up to 10MB), receive a short unique URL
• Read a snippet by short URL
• Optional expiration (1 hour, 1 day, 1 week, never)
• No authentication required (anonymous snippets)
• Optional: syntax highlighting, view count

Non-functional:

• Scale: 1M new snippets/day, 100M reads/day (assume 100:1 read:write ratio — Pastebin-like services have very high read skew)
• Availability: 99.9%
• Read latency < 100ms (this is a read-heavy service; reads must be fast)
• Write latency < 500ms (writing is infrequent; users tolerate it being slower)
• Snippets are immutable after creation (simplifies caching — no invalidation)
• Durability: snippets must not be lost (user shared this URL publicly)

Scope out: User accounts, editing, versioning, collaboration. Focus on create and read.

Step 2: Estimate Scale

Writes: 1M snippets/day / 86,400 = ~12 writes/second
Reads:  100M reads/day / 86,400 = ~1,160 reads/second
Peak (3× average): 36 writes/s, 3,500 reads/s

Storage:
  Average snippet: 20KB (10MB max, most are small code snippets)
  1M snippets/day × 20KB = 20 GB/day = 7.3 TB/year
  At 20KB average, this is primarily text — compressible.
  With 3:1 compression: ~2.4 TB/year

Short URL generation:
  1M snippets/day × 365 days × 10 years = 3.65 billion snippets
  6-character base62 URL: 62⁶ = 56.8 billion combinations → sufficient for 10+ years

Observations from the numbers:

• Writes are modest (12/s). Any database handles this.
• Reads are moderate (1,160/s), but with 100:1 skew — caching will be very effective.
• The key challenge is unique short URL generation (avoiding collisions) and efficient read-by-short-URL.
• Storage is large (TBs over years) but snippets are immutable — perfect for object storage.

Step 3: Data Model

Snippet:
  short_url    VARCHAR(8)  PRIMARY KEY  (e.g., "aB3xK9")
  content_key  VARCHAR(256)             (S3 object key — where the content lives)
  content_size BIGINT                   (bytes)
  created_at   TIMESTAMP
  expires_at   TIMESTAMP (nullable)
  content_type VARCHAR(64)              (text, python, javascript, etc.)
  view_count   BIGINT

Notice: the content itself is NOT in the database row. It's in S3. The row is metadata — fast to read, small (< 200 bytes per row). The actual text content is fetched from S3 (or a cache in front of S3) separately. This separation is intentional: keeping hot metadata (URL → location mapping) in a fast relational database while cold content lives in cheap object storage.

Step 4: Core Flows

Write path:

Read path:

Short URL collision handling (hash approach): If two different snippets produce the same 8-hex-char hash prefix, the second write queries the database, sees the short URL is taken, compares full hashes — if different, append one character and retry. In practice: 2³² ≈ 4 billion values; with 3.65 billion total snippets over 10 years, collision probability grows but the retry mechanism handles it cleanly.

Step 5: Storage and Scaling

PostgreSQL:
  Stores snippet metadata (~200 bytes/row × 3.65B rows ≈ 700 GB over 10 years)
  Single primary handles 12 writes/second easily.
  Two read replicas for redundancy and read scaling.
  Index: PRIMARY KEY on short_url (8-byte hash lookups → B-Tree, fast).
  Expired snippets: daily job deletes rows where expires_at < NOW() (Chapter 3: careful with large deletes — don't lock the table; use batched deletes with LIMIT 1000).

Redis:
  Two-tier cache:
    1. short_url → metadata (small, ~300 bytes, TTL = expiration time)
    2. short_url → content (large, up to 10MB per entry — use carefully)
  For content: only cache snippets < 100KB (small snippets are read most). Large snippets served from S3 directly.
  Estimate cache size: 100M reads/day × 80% cache hit rate → 20M cache misses/day.
  If 20% of snippets account for 80% of reads (Pareto), caching top 20% of snippets by access:
    3.65B × 20% = 730M snippets. At 20KB average and 3:1 compression → ~4.9 TB. Too large for Redis.
  Revise: cache only snippets accessed in last 24 hours. At 1M new/day, cache last 7 days' snippets: 7M × 20KB = 140 GB. Feasible for a Redis cluster.

S3:
  All snippet content stored as objects. Key: {content_hash}.txt
  S3 handles TBs natively — infinite scale for this use case.
  S3 Standard storage class for recent snippets; S3 Glacier for snippets older than 1 year (3× cheaper).
  S3 charges per request (~$0.0004/1000 GETs). At 100M reads/day, cache eliminates ~80% → 20M S3 GETs/day → $8/day. Acceptable.

Step 6: Failure Modes

Redis failure: Reads fall back to PostgreSQL + S3. Latency increases from ~1ms to ~55ms for cache misses. The system degrades gracefully — not down, just slower. The database handles the load until Redis recovers (PostgreSQL + replicas handle ~1,160 reads/second easily).

S3 outage: Content unreadable, metadata readable. Show users "content temporarily unavailable" instead of 404. Do not serve stale content incorrectly identified as current. S3 is 11-nines (99.999999999%) durable — practically never the failure point.

Short URL collision rate increase: Monitor collision retries as a metric (Chapter 18). Alert if retry rate exceeds 0.1% of writes. Remediation: switch from 8-char to 10-char hash prefix, expanding the address space from 2³² to 2⁴⁰.

Expiration: A background worker runs every minute, deletes expired rows in PostgreSQL in batches, and deletes S3 objects. Redis TTLs handle cache expiration automatically. Eventual consistency: a snippet may be readable from Redis for up to its TTL after expiration — acceptable given the use case.

Part 6: Patterns That Appear Everywhere

Across all system design problems, a small set of patterns recur. Recognising them accelerates your reasoning.

Fan-out: One event triggers many writes. Twitter tweet → many feed updates. Email sent → many inbox updates. Message in a group chat → N member notifications. Always evaluate: how many downstream writes does one upstream write trigger? If it's large, fan-out must be async (message queue).

CQRS (Command Query Responsibility Segregation): Separate the write model (optimised for writing) from the read model (optimised for reading). Twitter's write model is a normalised tweet table. The read model is pre-computed feed lists in Redis. They're different data structures, updated asynchronously. This is CQRS at its core. Any time your read and write access patterns are fundamentally different, consider CQRS.

Event sourcing: Instead of storing current state, store the log of events that produced the state. The current state is derived by replaying the log. Kafka's log-compaction (Chapter 15) is event sourcing. Useful when you need audit trails, time travel (what was the state at T?), or multiple projections from the same events (read models for different use cases).

Read-your-writes consistency: After a user writes, their own subsequent reads must see that write. Handled by routing the user's reads to the primary (not replicas) for a short window after a write, or by caching the write result locally and serving it from cache while replication catches up (Chapter 4, 6).

Saga pattern (distributed transactions): When a business operation spans multiple services and each step might fail (Chapter 13). Design each step with an idempotent compensating action. Sequence: reserve inventory → charge payment → ship order. Failure in ship: refund payment (compensating action), release inventory (compensating action). The saga replaces a distributed transaction that would require 2PC (with its blocking and coordination overhead).

Back-pressure: When a producer generates work faster than a consumer can process it, the consumer pushes back on the producer to slow down rather than letting the queue grow unbounded (Chapter 14). Essential for any system with a queue — without back-pressure, the queue grows until it exhausts memory or disk, causing a cascade failure.

Vocabulary Map

Practice Problems

Problem 1 — Failure Analysis

A team designs a URL shortener using the following architecture: all snippets stored in a single PostgreSQL instance (no replicas). Reads query the database directly (no cache). Short URLs are generated by auto-incrementing integer IDs converted to base62.

Calculate: at 100M reads/day, what is the read load on the single PostgreSQL instance in reads/second at average and 3× peak? A PostgreSQL instance with SSD storage handles approximately 5,000 simple indexed reads/second before latency degrades. At what point does this design fail?

Identify two additional problems: (a) what does auto-incrementing base62 reveal about the system that a hash-based URL wouldn't? (b) what happens to all short URLs if the single PostgreSQL instance's disk fails with no replicas and no backups? Quantify the data loss.

Redesign the read path to handle 100M reads/day with < 100ms P99 latency and survive single-instance PostgreSQL failure. Specify: the caching layer (what goes in cache, what TTL, what eviction policy), the replication strategy (how many replicas, sync or async and why — reference Chapter 4 and 5), and what happens to reads during a primary failure (read from replica? accept stale data for how long?).

Problem 2 — Design from Scratch

Design a distributed rate limiter as a standalone service (not embedded in each application). The interface: check_and_consume(api_key, limit, window_seconds) → (allowed: bool, remaining: int, reset_at: timestamp). Multiple application servers call this service before processing each request.

This service must: handle 500,000 calls/second at peak, add < 2ms P99 latency to each request, survive a single server failure without dropping rate limit state, and correctly enforce limits even when called from 50 concurrent application servers.

Apply the full framework: functional and non-functional requirements (the problem statement gives most, derive the rest), estimate load on the rate limiter service itself, choose data storage (Chapter 17 covered Redis for rate limiting — what else could work here? trade-offs?), design the read/write path for check_and_consume, handle the distributed consistency problem (two servers calling check_and_consume simultaneously for the same api_key), and address the failure mode: what happens when the rate limiter service itself goes down? (fail open vs. fail closed — Chapter 17 — and why your answer depends on what the rate limiter is protecting).

Problem 3 — Synthesis

Design the notification system for a social platform (Twitter-like). When user A posts a tweet, all of A's followers who have notifications enabled should receive a push notification within 10 seconds.

Constraints: 100M DAU, average 200 followers per user who follow with notifications enabled, celebrities up to 50M followers, 580 tweets/second (from the Twitter estimation). Push notification services (APNs for iOS, FCM for Android) accept at most 1,000 notifications/second per API key, with rate limiting enforced by the provider.

Connect to: Chapter 14 (message queue for async fan-out), Chapter 17 (rate limiting the outbound calls to APNs/FCM), Chapter 15 (Kafka for the fan-out pipeline), Chapter 16 (load balancing across notification workers), Chapter 18 (what metrics tell you the notification pipeline is healthy). Design the full system and explicitly name which concepts from which chapters apply at each component.

Problem 4 — Go Deeper

The Pastebin design from the chapter uses content-addressable storage: the short URL is derived from a hash of the content. This means two users who paste identical content get the same short URL.

(a) Is this a privacy problem? If user A pastes a private API key and user B pastes the same key (maybe they share it), they get the same URL. Anyone who knows the URL can see the content. Does the hash reveal anything about whether the content is popular (i.e., many people have pasted the same thing)?

(b) Design an alternative that provides privacy isolation: each paste gets a unique URL regardless of content, but identical content is only stored once in S3 (content deduplication for storage efficiency). The mapping: unique_short_url → content_hash → S3 object. How does this change the data model and write path? Draw the new entity relationship. What is the storage overhead compared to the content-addressable approach?

(c) How would you design expiration in this system where multiple short URLs point to the same S3 object? If user A's paste expires but user B's paste (with the same content) hasn't, should the S3 object be deleted? What data structure and what process handles this correctly? This is a reference-counting problem — connect it to Chapter 10's garbage collection discussion (Cassandra gc_grace_seconds) and Chapter 13's Saga pattern for the delete operation.

Problem 5 — Connect to Previous Chapters

You're designing a leaderboard system: a real-time ranking of the top 100 players by score in a multiplayer game. 10M active players. Scores update continuously — at peak, 50,000 score updates per second. Any player can query "what is my current rank?" and "show me the top 100 players."

(a) What data structure stores the global ranking? (Hint: Chapter 9 discussed Redis data structures — which one gives you O(log N) rank queries and O(log N) score updates for N players?) What is the memory requirement for 10M players with O(log N) per player?

(b) At 50,000 updates/second, all going through a single Redis sorted set: what is the bottleneck? Redis is single-threaded for commands — what is the maximum throughput of ZADD operations on a single Redis instance? (Look up Redis throughput benchmarks from memory: ~100,000 simple commands/second on a single instance.) Is 50,000 ZADD/second feasible?

(c) If the single Redis sorted set can't handle 50,000 updates/second, propose a sharding strategy for the leaderboard. You can shard by player ID — but then "top 100 globally" requires merging the top 100 from each shard (scatter-gather — Chapter 7). What is the complexity of this merge, and how often must it run? Design the eventual consistency trade-off: how stale can the global top 100 be? 1 second? 10 seconds?

(d) Connect to Chapter 13: a score update and a rank read happening simultaneously must be consistent — you shouldn't read a rank that reflects a score that hasn't committed yet. But with the sharded leaderboard and async merging, eventual consistency is unavoidable. Define the exact consistency model this system provides (using vocabulary from Chapter 6), and identify when this matters and when it doesn't.

What's Coming Next

Chapters 20–24 are design problems — complete system design exercises worked from blank problem statement to full architecture, applying this framework and every tool from the preceding 18 chapters. Each chapter is a different problem domain: a real-time messaging system, a distributed key-value store, a global CDN, a payment processing system, and a search engine. These are the hardest FAANG interview questions, answered from first principles. The framework from Chapter 19 is the skeleton — the preceding chapters are the flesh. You are ready to attempt them.

Chapter 19 complete. This chapter is the pivot point of the curriculum — everything before it built the tools, everything after applies them. The single most important habit this chapter installs: write the requirements and scale estimates before drawing any boxes. Engineers who skip this step produce architectures that are either over-engineered (built for scale they'll never reach) or under-engineered (no caching for a 100:1 read-heavy workload). The numbers tell you which problems are real. Problem 3 (notification system) is the most comprehensive synthesis problem yet — it requires connecting seven prior chapters to design one coherent system. Do it on paper before reading ahead.