Chapter 22: Design a Global CDN

Curriculum: Distributed Systems from First Principles Chapter: 22 of 24 Part: 10 — Synthesis Design Problems Framework: Chapter 19's six-step process, applied in full Chapters exercised: 1 (speed of light, latency numbers), 6 (consistency models, CAP), 7–8 (sharding, consistent hashing), 9 (caching — every concept), 11–12 (consensus, Raft), 14–15 (message queues, Kafka), 16 (load balancing, anycast), 17 (rate limiting, DDoS), 18 (observability at scale) Estimated reading time: 80–90 minutes

The Problem Statement

Design a Content Delivery Network that serves static content — images, videos, JavaScript bundles, CSS, HTML pages — to users anywhere on earth with sub-50ms latency. The CDN must handle 10 petabytes of data transfer per day, cache content at hundreds of globally distributed edge locations, invalidate cached content within 30 seconds of a customer update, protect origin servers from traffic spikes and DDoS attacks, and support video streaming at adaptive bitrates. Design for Cloudflare or Akamai scale.

Opening: The Physics Problem CDNs Solve

Before requirements, the fundamental constraint this entire design is solving: the speed of light.

Light (and electrical signals in fiber optic cables) travels at approximately 200,000 km/second in glass fiber (accounting for the refractive index — slower than vacuum). The earth's circumference is ~40,000 km. New York to London: ~5,500 km. The minimum possible one-way latency for an electrical signal NYC → London is:

5,500 km / 200,000 km/s = 27.5ms one-way
Round-trip minimum: 55ms

Actual measured RTT NYC → London: ~75ms
(Fiber doesn't follow straight lines. Plus routing overhead.)

NYC → Tokyo: ~8,000 km → 40ms one-way → 80ms RTT minimum → ~120ms actual
NYC → Mumbai: ~12,500 km → 62.5ms one-way → 125ms RTT minimum → ~180ms actual

Chapter 1 established that humans notice latency above ~100ms and that page load time directly affects user engagement and revenue (Amazon's study: 100ms additional latency costs 1% of sales). If your origin server is in Virginia and a user is in Mumbai, every asset load has a 180ms RTT floor — before any server processing. Loading 30 assets for a typical webpage: 30 × 180ms = 5,400ms minimum even with perfect HTTP pipelining. Users in Mumbai see a broken, unusable web.

A CDN solves this by placing servers — edge nodes or Points of Presence (PoPs) — in data centers physically close to users. A PoP in Mumbai serves requests from a Mumbai user with ~5ms RTT instead of ~180ms. The 36× latency reduction is purely physical — getting the data closer to the user.

The secondary benefit: cost. Serving 10 PB/day from a single origin data center requires 10 PB/day × $0.08/GB (typical egress pricing) = $800,000/day in bandwidth costs. A CDN caches content at the edge — the origin pays egress costs only for cache misses, which are a fraction of total requests. CDNs reduce origin egress costs by 80–99% for cacheable content.

Step 1: Requirements Clarification

Functional Requirements

1. Serve cached content from the edge. Static assets (images, CSS, JS, fonts, video segments) are served from the geographically closest PoP. If the PoP has the content cached, it responds directly without contacting the origin.
2. Origin pull on cache miss. If the PoP doesn't have the content, it fetches it from the origin server, caches it locally, and serves it to the user.
3. Cache invalidation. When a customer updates content on their origin, the CDN must expire cached copies globally within 30 seconds.
4. SSL/TLS termination at the edge. The TLS handshake is expensive (1–2 additional RTTs). Terminating TLS at the edge PoP means the expensive handshake happens over a ~5ms local connection, not a ~180ms cross-ocean connection.
5. Video streaming support. Serve HLS (HTTP Live Streaming) and DASH segments — small chunks of video (2–10 seconds each) that clients fetch sequentially. Adaptive bitrate switching: serve different quality segments based on the client's measured bandwidth.
6. DDoS mitigation. Absorb volumetric DDoS attacks (100+ Gbps) at the edge, preventing them from reaching origin servers.
7. CDN analytics. Customers can see cache hit rates, traffic volume, latency percentiles, and error rates for their content.

Non-Functional Requirements

• Scale: 10 PB/day data transfer. ~116 GB/s sustained average throughput. Peak (3× average): 350 GB/s.
• Latency: P99 cache hit response time < 50ms for any user on earth (edge must be within ~5,000 km of every significant population center). Cache miss adds the origin fetch time (not bounded by CDN — up to the customer's origin performance).
• Cache hit rate: > 90% globally for a typical static-asset workload. (10% of requests reach the origin.)
• Availability: 99.99% per PoP. Global availability: a PoP outage affects only users closest to that PoP; they fall back to the next-closest PoP.
• Invalidation latency: Cache purge propagates to all PoPs within 30 seconds of the API call.
• PoP count: 200+ PoPs globally, covering major population centers on all continents.

Step 2: Estimate Scale

Data transfer:
  10 PB/day = 10 × 10¹⁵ bytes / 86,400s = ~116 GB/s average
  Peak: 350 GB/s

Per-PoP load (200 PoPs, traffic distributed by geography):
  Average per PoP: 116 GB/s / 200 = 580 MB/s
  Peak per PoP: 1.75 GB/s
  A single 100GbE NIC: 12.5 GB/s — each PoP needs 1 NIC at average load.
  Large PoPs (NYC, London, Tokyo) handle 10–50× average: need 10–20 servers each.

Requests per second:
  Average object size: 100 KB (mix of images, JS bundles, video segments)
  116 GB/s / 100 KB = 1.16 million requests/second globally
  Per PoP average: 5,800 requests/second
  Large PoPs: 50,000–200,000 requests/second

Cache storage per PoP:
  At 90% cache hit rate and 10 PB/day:
    Unique content served/day ≈ 1 PB (10% of 10 PB is unique fetches)
    But content reuse across days: working set is much smaller
    Typical CDN working set: ~50 TB of frequently accessed content
    Per PoP cache: 500 GB–5 TB (SSDs — fast random reads for cache lookup)
    Large PoPs: 10–50 TB NVMe SSD storage

Cache invalidation events:
  Typical: 100,000 invalidation requests/day from all CDN customers
  Each invalidation must propagate to 200 PoPs
  100,000 × 200 = 20M invalidation events/day → 231 events/second across the system
  Burst: a customer deploying a new version invalidates 10,000 URLs simultaneously
  10,000 URLs × 200 PoPs = 2M invalidation events in seconds

The key observation: the CDN is read-dominated (all traffic is reads). The only writes are cache invalidations (rare) and cache population on miss (moderate). This is the most read-heavy system in the curriculum — perfect for a multi-tier cache hierarchy.

Step 3: The PoP Architecture — Inside One Edge Node

Before designing the global system, understand what happens inside a single PoP. A PoP is a data center (leased or owned) with several hundred servers and high-bandwidth network connections to local ISPs.

TLS Termination — Why the Edge Matters

A TLS 1.2 full handshake requires 2 RTTs before the first byte of content is transferred. TLS 1.3 reduces this to 1 RTT. TLS 1.3 with 0-RTT session resumption: 0 additional RTTs (the client sends the request in the first packet, encrypted with a previously established session key).

Without CDN (user in Mumbai, origin in Virginia):
  DNS lookup: 180ms
  TCP connect: 180ms (1 RTT)
  TLS 1.3 handshake: 180ms (1 RTT)
  HTTP request → response: 180ms (1 RTT)
  Total to first byte: 720ms

With CDN (PoP in Mumbai, ~5ms RTT):
  DNS lookup: 5ms (answered by CDN's authoritative DNS at the edge)
  TCP connect: 5ms
  TLS 1.3 handshake: 5ms
  HTTP request → response: 5ms (cache hit — no origin fetch)
  Total to first byte: 20ms

Improvement: 36× latency reduction for first byte.

The Two-Tier Cache Within a PoP

Each cache server has two cache levels:

L1 — In-process memory (4 GB, LRU eviction): Holds the hottest ~40,000 objects (at ~100KB average). An LRU hash map: O(1) lookup, O(1) eviction. This cache is served from the CPU's L3 cache and RAM — Chapter 1's sub-microsecond memory access times. Response times for L1 hits: <1ms.

L2 — Local NVMe SSD (2 TB): Holds ~20 million objects. Chapter 1's SSD random read: ~100μs. Response times for L2 hits: ~1–5ms. The SSD cache is implemented as a log-structured store (Chapter 10's LSM insight applies here too: write new cached objects sequentially, read by index, compact by evicting cold objects). Sequential writes to SSD are 10–100× faster than random writes — the log-structured approach maximises write throughput for cache population.

L3 — Peer cache within the PoP: If neither L1 nor L2 has the object, before going to origin, ask sibling cache servers in the same PoP. This uses ICP (Internet Cache Protocol) or a modern equivalent. If any sibling has the object, serve it peer-to-peer within the PoP data center. Within-datacenter RTT: ~0.5ms. This avoids the origin fetch for objects that are cached on a different server in the same PoP (each server holds 1/N of the PoP's cached objects; the peer cache collapses N servers into one logical cache).

Consistent hashing within the PoP (Chapter 8): The load balancer uses consistent hashing on the URL to route requests for the same URL to the same cache server. This maximises cache hit rates per server — if all requests for /images/logo.png go to Server 3, Server 3 caches it once and serves it forever. Without consistent hashing (round-robin load balancing), /logo.png might be cached on all N servers simultaneously, wasting N× the cache storage and reducing effective cache capacity by N×.

Step 4: Global Routing — How Users Reach the Closest PoP

The CDN serves a single hostname (e.g., cdn.example.com) from 200 PoPs. How does a user in Mumbai get routed to the Mumbai PoP instead of the Virginia PoP?

Anycast

Anycast (Chapter 16's global load balancing section) is the primary mechanism. Every PoP announces the same IP address block (e.g., 104.21.0.0/20) via BGP to neighboring ISPs. BGP routes packets to the "nearest" announcement based on AS-hop count. A user in Mumbai whose ISP peers with the CDN's Mumbai PoP gets routed to Mumbai. A user in London whose ISP peers with the London PoP gets routed to London. Same destination IP, different physical servers.

Anycast is the correct routing mechanism for UDP-based protocols (DNS, QUIC) where a request is a single packet. For TCP (HTTP/1.1, HTTP/2), there's a subtlety: if the user's route changes mid-connection (BGP route flap), their TCP connection is broken. For this reason, CDNs use anycast to route the initial DNS lookup and the first connection to the right PoP, then the TCP connection stays on that PoP for its duration.

DNS-Based Routing

For customers who cannot use anycast (e.g., serving dynamic content where session state matters), the CDN's authoritative DNS server returns a different A record based on the requesting resolver's IP:

This is GeoDNS (Chapter 16). Less accurate than anycast (it routes by resolver location, not client location, and many resolvers are far from their clients — e.g., Google's 8.8.8.8). But it works for TCP-sensitive routing.

Fallback PoP Selection

If the nearest PoP is overloaded or experiencing an outage: the anycast routing naturally failsover. If Mumbai PoP withdraws its BGP route announcement (because it detects a failure), BGP reconverges and Mumbai users' packets are now routed to the next-closest PoP (e.g., Singapore, ~5 hops away instead of 2). BGP reconvergence time: 30–90 seconds. During reconvergence, users may experience elevated latency or connection failures.

The CDN actively monitors PoP health and proactively withdraws BGP routes from degraded PoPs — a faster failover than waiting for BGP to detect the route as unreachable.

Step 5: The Cache Hierarchy — Edge, Regional, Origin Shield

A two-layer cache hierarchy dramatically reduces origin traffic:

The origin shield prevents the "thundering herd": if 200 PoPs all miss simultaneously for the same URL, without an origin shield they all hit the origin simultaneously. With origin shield, all 200 misses collapse into 1 origin request.

The thundering herd problem (Chapter 9 revisited): A new URL is first requested by 10,000 users simultaneously (e.g., a viral image). None of the edge PoPs have it. Without origin shield, all 200 PoPs forward their miss to the origin: the origin receives 200 simultaneous requests for the same object. With origin shield: all 200 edge PoPs forward to the regional cache, which forwards to the origin shield, which collapses all 200 into one request. The origin receives exactly 1 request. This is Chapter 9's stampede problem and its solution — request coalescing — applied at global scale.

Request coalescing within a PoP: If 1,000 users request the same uncached URL simultaneously, the PoP's cache server sends one request to origin and holds the other 999 requests waiting. When the origin responds, all 1,000 are served simultaneously. This is sometimes called "request deduplication" or "coalescing." It's essential — without it, a viral URL would generate 1,000 simultaneous origin requests per PoP × 200 PoPs = 200,000 concurrent origin requests for one URL.

Step 6: Cache Invalidation — The Hardest Problem

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

At CDN scale, cache invalidation must propagate to 200 PoPs in 30 seconds. This is a distributed systems coordination problem.

What Customers Control: Cache-Control Headers

The CDN respects the HTTP Cache-Control header on origin responses:

Cache-Control: public, max-age=31536000, immutable
  → Cache forever (1 year). Content will not change. (Used for versioned assets: /static/app.a3f8b.js)

Cache-Control: public, max-age=300
  → Cache for 5 minutes. After 5 minutes, re-validate with origin.

Cache-Control: public, max-age=86400, stale-while-revalidate=3600
  → Cache for 1 day. For the next 1 hour after expiry, serve stale content while
    asynchronously fetching the new version. (Chapter 9's stale-while-revalidate)

Cache-Control: no-store
  → Do not cache. Every request hits origin. (Used for user-specific data)

Best practice: Version your static assets. Instead of /logo.png (which must be invalidated on change), use /logo.a3f8b214.png (the hash of the content is in the filename). The versioned URL is immutable — the CDN can cache it forever. When the logo changes, the new version gets a new hash URL. The HTML references the new URL. No CDN invalidation needed. This is why webpack and other build tools generate content-hash filenames.

Explicit Cache Purge API

For assets that cannot be versioned (HTML pages, /robots.txt, /favicon.ico), customers call the CDN's purge API:

POST /cdn/v1/purge
{
  "urls": ["https://cdn.example.com/index.html",
           "https://cdn.example.com/product/42"]
}
→ Purge accepted. Propagation SLA: 30 seconds.

Invalidation Propagation — The Design Problem

The purge request arrives at one CDN control-plane server. It must reach all 200 PoPs within 30 seconds. This is a fan-out problem identical to Chapter 20's message fan-out.

Option 1: Direct API call to each PoP's cache servers

For 200 PoPs × 10 cache servers per PoP = 2,000 simultaneous API calls. At 50ms per call: 50ms to fan-out (in parallel). Sounds fast. But: each cache server has its own in-memory and SSD cache. Purging requires each server to delete the URL from its cache index. At 2,000 simultaneous calls for a single purge event: this works at current scale. But at 100K purge events/second, 100K × 2,000 = 200M API calls/second — not feasible.

Option 2: Pub/Sub via a distributed message bus

The control plane publishes purge events to a Kafka topic (Chapter 15) partitioned by cache_key. Each PoP has a Kafka consumer that reads purge events for its region. The cache servers within the PoP listen to the consumer and delete the specified URLs from their caches.

Kafka provides durable, ordered delivery with replay capability. If a PoP is down during a purge, when it recovers it reads from its Kafka consumer offset and applies all missed purge events before rejoining the serving pool. This is Chapter 15's "log as durable event store" applied to cache invalidation.

Option 3: Versioned cache keys (no explicit purge)

Cloudflare's preferred approach: every cache entry includes a version token (a hash of the customer's current "cache generation"). When the customer purges all their content, the control plane increments the generation hash. The new hash is broadcast to all PoPs. PoPs include the generation hash when checking their cache: a cached entry with an old generation hash is treated as a miss. No explicit deletion — old entries naturally expire via LRU or TTL. Purge propagation = propagating one small hash value to 200 PoPs (trivial).

This approach is elegant but requires that cache keys include the generation hash, which means changing the cache key design. Fastly and Cloudflare both use variants of this approach.

Step 7: Video Streaming — HLS and Adaptive Bitrate

Video is the dominant CDN workload by bytes: Netflix alone accounts for ~15% of global internet traffic. Understanding how CDNs serve video from first principles reveals why the protocol design matters.

HLS — HTTP Live Streaming

HLS (Apple's HTTP Live Streaming, now an IETF standard) works by splitting a video into small segments:

video.mp4 (2-hour movie, 8 GB at 1080p)
  → Segmented into 2-second chunks
  → 3,600 segments × 2.2 MB each = ~8 GB total

Manifest file (video.m3u8):
  #EXTM3U
  #EXT-X-VERSION:3
  #EXT-X-TARGETDURATION:2
  #EXTINF:2.0,
  segment-001.ts
  #EXTINF:2.0,
  segment-002.ts
  ...
  #EXTINF:2.0,
  segment-3600.ts
  #EXT-X-ENDLIST

The client (browser, TV app) fetches the manifest file first. Then it fetches segments sequentially, each as a separate HTTP GET request. Each segment is a regular HTTP response — fully cacheable by the CDN.

Why this is brilliant for CDNs: Each 2-second segment is an immutable file. The CDN can cache it forever (max-age=31536000). When Netflix's origin encodes a new segment (for a live stream), it gets a new URL. Caching is trivial. Cache hit rates for popular content approach 100% — once the first viewer requests segment-001.ts from a PoP, every subsequent viewer gets it from the PoP's cache without touching Netflix's origin.

Adaptive Bitrate (ABR) Streaming

The same video is encoded at multiple quality levels:

For the same video, multiple renditions:
  240p (low):   250 Kbps → segment size: 62.5 KB per 2-second chunk
  480p (medium): 1 Mbps  → segment size: 250 KB per 2-second chunk
  720p (high):   3 Mbps  → segment size: 750 KB per 2-second chunk
  1080p (HD):    8 Mbps  → segment size: 2 MB per 2-second chunk
  4K (ultra):   25 Mbps  → segment size: 6.25 MB per 2-second chunk

Manifest variant file (master.m3u8):
  #EXT-X-STREAM-INF:BANDWIDTH=250000,RESOLUTION=426x240
  240p/video.m3u8
  #EXT-X-STREAM-INF:BANDWIDTH=1000000,RESOLUTION=854x480
  480p/video.m3u8
  #EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720
  720p/video.m3u8
  ...

The client measures its download speed by timing how long each segment takes to download. If a 720p segment (750 KB) takes 3 seconds to download (250 KB/s bandwidth), the client switches to 480p for the next segment. If the connection improves, it switches back up. This is ABR (Adaptive Bitrate) — seamless quality switching without buffering.

CDN impact: Each rendition at each quality is a separate set of segments — all independently cacheable. Netflix's CDN (built in-house: Open Connect) pre-positions content on CDN nodes using a push model: before peak hours, popular content is proactively pushed to every PoP at every quality level. Cache hit rate approaches 100% during the push window. The pull model (origin pull on miss) would create thundering herds at 8pm when millions of users start the same episode simultaneously.

Live Streaming — The Cache TTL Problem

For live video, segments are generated in real time. Segment-001.ts is the first 2 seconds of the live broadcast — it becomes available as the broadcast starts. The manifest file changes every 2 seconds (new segments are added). This creates a conflict with CDN caching:

Live manifest changes every 2 seconds:
  Cache-Control: max-age=2  (cache for only 2 seconds)
  
  At 1M concurrent viewers × 1 manifest fetch / 2 seconds = 500,000 requests/second
  For the manifest alone — all of which need near-fresh data.
  
  Cache TTL of 2 seconds with request coalescing:
  A PoP receives 1,000 requests/second for the live manifest.
  Within each 2-second window: 2,000 requests, served from cache (1 origin fetch per 2s).
  Without coalescing: 2,000 origin requests per 2 seconds — the origin is overwhelmed.

Live streaming requires aggressive request coalescing at the edge (collapse the 2,000 manifest requests into 1 origin fetch per TTL window) and short TTLs (2–10 seconds). CDNs handle this with a special "streaming cache" mode that coalesces requests and applies TTLs as short as 1 second.

Step 8: DDoS Mitigation at the Edge

A CDN's globally distributed architecture is inherently a DDoS mitigation tool: instead of one 100 Gbps pipe to the origin, attackers must overwhelm 200 PoPs each with 1.75 Gbps capacity — much harder. But large DDoS attacks (terabit-scale) can still overwhelm individual PoPs. CDNs have dedicated mechanisms.

Volumetric DDoS (L3/L4)

A botnet sends 500 Gbps of UDP packets to the CDN's anycast IP. Individual PoPs absorb the fraction of traffic routed to them by BGP (traffic is naturally distributed by anycast). A PoP receiving 5 Gbps of junk UDP:

At the NIC level (before the kernel):
  eXpress Data Path (XDP) / DPDK: process packets in kernel bypass mode.
  Rule: "DROP all UDP packets not on port 443 or 80 to our IP ranges."
  Throughput: 100+ Gbps line rate dropping. CPU is not involved.
  (Chapter 16's L4 load balancer operates here too)

XDP/DPDK packet processing can drop 100+ Gbps of traffic per server without kernel involvement — the NIC itself (or a dedicated FPGA) applies simple filtering rules at wire speed. For a 200-PoP network with 10 Gbps average capacity each: total scrubbing capacity is 2 Tbps. The largest DDoS attacks on record (~3.47 Tbps, 2023) are absorbed by the network's sheer distributed capacity.

Application Layer DDoS (L7)

More sophisticated: the attacker sends valid HTTPS requests (expensive — requires TLS handshake), but targets expensive endpoints (slow search queries, large file uploads) to exhaust the CDN's compute rather than bandwidth.

CDN-level rate limiting (Chapter 17): per-IP, per-subnet, per-ASN limits on request rates. An IP sending 10,000 requests/second is blocked at the edge before reaching cache or origin. The rate limiter uses the sliding window counter (Chapter 17) in a Redis cluster shared across all cache servers in a PoP.

CAPTCHA challenges: The CDN can inject a CAPTCHA challenge page for IPs that look suspicious (new IP, unusual request pattern, high rate) — served from the CDN itself, not the origin. Only CAPTCHA-passing clients get their requests proxied to origin.

Bot detection: Machine learning models running at the edge classify requests as "bot" or "human" based on: TLS fingerprint (JA3 hash — the cipher suite order reveals the TLS library, which reveals the client), HTTP/2 stream priority patterns (browser behavior is distinctive), request timing patterns, header ordering. Cloudflare's Bot Score is exactly this: a per-request score from 0 (bot) to 100 (human), computed at the edge in <1ms.

Step 9: The Origin Shield as a Reverse Proxy

The origin shield is not just a cache — it is a full-featured reverse proxy that sits between the CDN's regional caches and the customer's origin server:

Regional cache miss → Origin Shield → Customer origin

Origin Shield functions:
  1. Request coalescing: collapse 200 simultaneous misses to 1 origin request
  2. Connection pooling: maintain persistent HTTP/2 connections to origin
     (Avoid TCP + TLS handshake overhead on every cache miss)
  3. Health checking (Chapter 16): monitor origin, remove unhealthy origin servers
  4. Load balancing: if origin has multiple servers, balance across them
  5. Retry logic: on origin 5xx, retry up to 2 times before returning error to edge
  6. Response buffering: buffer the full origin response before caching,
     so the origin connection is released quickly
  7. Compression: gzip/brotli compress responses that origin didn't compress

The origin shield solves the "thundering herd" problem definitively: with a single origin shield per customer, no matter how many PoPs miss simultaneously, the origin receives at most 1 concurrent request per unique URL. This is essential for viral content: when a YouTube video goes viral, every PoP worldwide misses on the first request — without origin shield, the origin would receive 200 simultaneous requests. With origin shield: 1.

Step 10: Security — TLS at Scale

The CDN terminates TLS for hundreds of thousands of customer domains (SNI — Server Name Indication allows one IP to serve TLS for many domains). Managing TLS certificates at this scale:

Certificate issuance: Each customer domain (e.g., cdn.customer.com) needs a TLS certificate. At 1 million customer domains, this means 1 million certificates. The CDN integrates with Let's Encrypt's ACME protocol for automated certificate issuance and renewal (Let's Encrypt issues certs in 90-second response time, for free, via automated domain validation). Certificate auto-renewal runs 30 days before expiry.

Certificate distribution: A newly issued certificate must reach all 200 PoPs. This is a distributed configuration update — similar to cache invalidation. Certificates are stored in a central key store (encrypted at rest), distributed to PoPs via an authenticated config delivery pipeline. Delay: ~10 seconds (the time for the distribution to propagate and for all cache servers in all PoPs to reload their TLS configs). This is why "SSL certificate is active" has a propagation delay after issuance.

TLS session tickets: TLS 1.3 session resumption uses session tickets — an encrypted blob issued by the server, stored by the client. On reconnect, the client presents the ticket and skips the full handshake (0-RTT or 1-RTT resumption). But: session tickets must be encrypted with a shared key, and the client may reconnect to a different cache server than the original. All cache servers in a PoP share a session ticket encryption key (rotated hourly, distributed via the config pipeline). A client connecting to Server 7 can resume its session on Server 3 because both share the same ticket key.

Step 11: Observability for a CDN (Chapter 18 Applied)

A CDN must answer real-time questions for customers: "What's my cache hit rate right now?" "Is my origin down?" "Where are users experiencing slow responses?" This requires per-PoP, per-customer, per-URL observability at petabyte scale.

The cardinality problem (Chapter 18 revisited): If you emit a metric per customer per URL: 1M customers × 100K URLs per customer = 100 billion time series. This destroys any metrics system. Instead, emit metrics aggregated per customer per PoP per minute:

cdn_requests_total{customer_id="cust123", pop="bom", status="HIT"}  (counter)
cdn_requests_total{customer_id="cust123", pop="bom", status="MISS"} (counter)
cdn_origin_latency_seconds{customer_id="cust123", pop="bom"}        (histogram)
cdn_bytes_transferred{customer_id="cust123", pop="bom"}             (counter)

1M customers × 200 PoPs × 4 metrics = 800M time series. Still high. Use a tiered approach: full granularity stored in a streaming analytics system (Kafka → ClickHouse or BigQuery) for customer dashboards. Aggregated metrics (per customer, global) in Prometheus for CDN health monitoring.

Real-time log streaming: Every cache request is a log event:

{
  "timestamp": "2024-01-15T10:23:45.182Z",
  "pop": "bom",
  "customer_id": "cust123",
  "url": "https://cdn.cust123.com/image.jpg",
  "cache_status": "HIT",
  "http_status": 200,
  "bytes": 102400,
  "ttfb_ms": 3,
  "client_ip": "203.0.113.1",
  "country": "IN",
  "asn": "AS24560"
}

At 1.16M requests/second globally: 1.16M log events/second. Kafka ingests these at the edge, fans them out to a columnar analytics store (ClickHouse, optimised for OLAP queries — Chapter 19's storage decision tree: analytical queries → columnar store). Customers query their logs via a web dashboard with 60-second latency (near-real-time). CDN infrastructure teams query for health monitoring with 10-second latency (Prometheus scraping per-PoP metric aggregators).

SLO for the CDN itself:

Cache hit rate SLO: > 90% globally, measured per hour.
  SLI: cdn_requests_total{status="HIT"} / cdn_requests_total
  Alert: if cache hit rate drops below 80% for any customer for > 5 minutes.

Origin error rate SLO: < 0.1% of cache misses result in origin errors.
  SLI: origin_errors_total / origin_requests_total
  Alert: burn rate > 14× → immediate page.

PoP availability SLO: 99.99% per PoP.
  Dead man's switch: if a PoP stops emitting metrics, alert immediately.
  (absent() in Prometheus — Chapter 18)

Step 12: The Complete Architecture

The control plane runs separately from the data plane:

Step 13: Failure Modes and Trade-Offs

PoP outage: Anycast BGP route withdrawn. Users fail over to next-closest PoP. BGP reconvergence: 30–90 seconds. During reconvergence: ~5% of requests for that PoP's users fail. After reconvergence: users are served from a more distant PoP (higher latency, but functional). Cached content at the failed PoP is lost — must re-warm from origin. Origin load temporarily increases.

Regional cache outage: Edge PoPs in the region fall back to direct origin pulls. Cache miss rate rises from 5% to 100% for the affected region. Origin traffic spikes 20×. The origin must be sized to handle this degraded traffic (Chapter 16's origin health checking). Origin shield prevents cascade: all misses are coalesced to origin shield, which rate-limits origin queries (Chapter 17) and queues excess requests rather than flooding origin.

Cache poisoning attack: An attacker tricks the CDN into caching a malicious response (e.g., by injecting a response with a forged Cache-Control header that makes the CDN cache the attacker's content for a long time, then serving it to all users). Mitigation: never cache responses from untrusted sources. Cache only origin responses from authenticated, TLS-verified origins. Validate Cache-Control headers (e.g., never cache Cache-Control: max-age=999999999). Implement a maximum cacheable TTL (e.g., 1 year).

Stale content served post-purge: A purge is issued but one PoP fails to apply it (Kafka consumer down). That PoP serves stale content until: (a) the object's natural TTL expires, (b) the Kafka consumer recovers and processes the purge event (from stored offset), or (c) the content is manually expired. This is why short TTLs (< 1 hour) are preferred for frequently changing content — the stale window is bounded even if invalidation fails.

The "cache everything" anti-pattern: A CDN misconfiguration caches authenticated, user-specific content (e.g., a Cache-Control: public response that actually contains personal data). User A's profile page is cached and served to User B. This is a catastrophic data privacy bug. The CDN must check for Set-Cookie headers (responses with cookies are typically user-specific — don't cache) and Authorization request headers (authenticated requests should not be cached by default). CDNs apply safe defaults and warn customers about misconfigurations.

Vocabulary Map

Practice Problems

Problem 1 — Failure Analysis

A customer deploys a new version of their website (index.html) and calls the CDN purge API. The purge propagates to 199 of 200 PoPs within 10 seconds. PoP Tokyo is experiencing a Kafka consumer outage — its consumer lag has grown to 500,000 events.

(a) What do Tokyo users see? For how long? (Kafka consumer processes ~10,000 events/second when healthy; backlog is 500,000 events → how long to process after recovery?)

(b) The customer's old index.html has Cache-Control: max-age=3600. The purge was issued 30 minutes after the new version was deployed. How long is the maximum window during which Tokyo users see the old content, in the worst case (Kafka consumer is down for 2 hours)?

(c) The customer switches their index.html to use Cache-Control: max-age=60, stale-while-revalidate=30. How does this change the failure mode? What is the maximum stale window in this configuration, with or without a purge?

(d) Redesign the customer's deployment workflow to eliminate the need for cache purge entirely, using content-hash URLs. Describe the full workflow: build step, HTML update, CDN caching strategy, and what happens when users have the old HTML cached in their browsers pointing to old content-hash URLs.

Problem 2 — Design from Scratch

Design the CDN's certificate management system for 1 million customer domains.

(a) Certificate issuance: a new customer adds their domain shop.newcustomer.com to the CDN. The CDN must obtain a TLS certificate from Let's Encrypt via ACME DNS-01 challenge (Let's Encrypt asks the CDN to add a TXT record to the customer's DNS to prove domain ownership). Design the flow: what systems are involved, what does the CDN's DNS infrastructure need to support, and how long does the end-to-end issuance take?

(b) Certificate storage and distribution: 1 million certificates × ~4 KB each = 4 GB of certificate data. Certificates must be available on every cache server in every PoP (200 PoPs × 100 servers = 20,000 servers). Design the storage and distribution architecture: where are certificates stored centrally, how are they pushed to PoPs, and what is the propagation latency to all 20,000 servers? What happens if a cache server starts serving a new customer's domain before the certificate has propagated?

(c) Certificate renewal: Let's Encrypt certificates expire after 90 days. At 1M certificates, how many renewals per day (uniform distribution)? What is the renewal rate per second? Design the renewal pipeline: when should renewal be triggered (30 days before expiry? 7 days?), how does the renewed certificate propagate to all PoPs, and what is the risk window if renewal fails?

Problem 3 — Synthesis

Connect the CDN to Chapter 9's caching failure modes and Chapter 12's consensus.

(a) Cache stampede at CDN scale: A celebrity tweets a link to an image (viral.jpg) hosted on a CDN. Within 1 second, 1 million users click the link. The image is not yet cached on any PoP (first-ever request). Calculate: how many PoPs receive requests simultaneously? How many simultaneous origin requests without request coalescing? How many with request coalescing (one per PoP)? How many with a properly configured origin shield (one globally)?

If the origin can handle 100 concurrent requests before experiencing degraded performance, which scenario(s) survive the viral surge? What is the maximum origin load for each scenario?

(b) The cache invalidation system uses Kafka partitioned by cache_key_hash. A customer issues 10,000 invalidations simultaneously (they redeployed their entire site). All 10,000 URLs hash to 10 Kafka partitions. Each partition has one consumer per PoP. How many consumers are processing simultaneously? (10 partitions × 200 PoPs = 2,000 consumers.) If each invalidation event takes 5ms to apply at one PoP, how long does it take all 200 PoPs to process all 10,000 invalidations? Is this within the 30-second SLA?

(c) A CDN operates globally across 5 regions. The CDN's control plane (purge API, certificate management, configuration) needs to be consistent — if a customer updates their origin URL in region US, the change must be seen by region EU before EU PoPs start routing traffic to the new origin. This requires distributed consensus across 5 regions. Should this use Raft (Chapter 12) or a simpler approach? What is the latency cost of Raft consensus across 5 globally distributed regions, given Chapter 1's inter-continental RTTs?

Problem 4 — Go Deeper

Analyse the economics of CDN caching and the "long tail" problem.

A CDN serves 1 billion unique URLs (aggregated across all customers). URL popularity follows a Zipf distribution: the most popular URL receives 1M requests/day, the 2nd most popular 500K, the Nth most popular 1M/N.

(a) The top 1,000 URLs account for what fraction of all requests? Compute: sum(1M/N for N=1 to 1000) / sum(1M/N for N=1 to 1B). Use the approximation: sum(1/N for N=1 to K) ≈ ln(K) + 0.577. What fraction of total traffic do the top 1,000 URLs represent?

(b) If each PoP has 2 TB of cache storage and the average object is 100 KB, the PoP can cache 20 million objects. Which objects should it cache? (The top 20M by request frequency — the hot working set.) Given the Zipf distribution, what fraction of total traffic is served from a 20M-object cache? (Compute sum(1/N for N=1 to 20M) / sum(1/N for N=1 to 1B).)

(c) The "long tail" problem: 999 million of the 1 billion URLs are requested so infrequently that they're never in cache when requested — every request is a cache miss (the object was evicted since the last request). For long-tail URLs, the CDN provides essentially no caching benefit — it's just a proxy. What is the minimum request frequency for a URL to achieve 50% cache hit rate on a cache with 2 TB capacity, 100 KB average object size, and LRU eviction? (Hint: a URL achieves 50% cache hit rate if it's requested more than once before being evicted.)

(d) Design an alternative caching strategy for the long tail: instead of caching long-tail objects at every edge PoP, only cache them at regional caches (higher capacity, fewer locations). Edge PoPs for long-tail requests act as proxies to the regional cache. The trade-off: long-tail requests have higher latency (edge → regional instead of edge → memory), but regional cache hit rate is higher (consolidating 200 PoPs' long-tail traffic into one regional cache means 200× more chances for the object to be warm). Analyse: if a URL receives 1 request/day globally distributed uniformly across 200 PoPs, what is the PoP-level cache hit rate? What is the regional cache hit rate? (Assume regional cache LRU with 200TB capacity — large enough that a URL is only evicted after 200 days of no access.)

Problem 5 — Connect to Previous Chapters

The CDN's cache invalidation system, DDoS mitigation, and video delivery each connect to specific prior chapters. For each scenario below, identify which chapter's mechanism applies and explain the connection precisely.

(a) A content creator releases a new movie on a streaming platform at 8pm. The CDN has pre-positioned all video segments at all PoPs (pushed during the day). At 8pm, 5 million users simultaneously open the movie. The first segment (segment-001-720p.ts) is cached at every PoP. The second user in each PoP serves from cache. Explain why the first request from each PoP does NOT trigger a thundering herd at origin, even though 200 PoPs each have a first request. Connect to Chapter 9's cache-aside pattern and the pre-positioning strategy.

(b) The DDoS mitigation system detects that IP 203.0.113.42 is sending 50,000 requests/second to a PoP (normal user rate: <10/second). The rate limiter blocks this IP. But the attacker switches to 10,000 different IPs (a botnet), each sending 5 requests/second — below the per-IP threshold. Design a solution using the multi-granularity rate limiting from Chapter 17: what rate limit dimension catches this attack? (Hint: per-IP misses it, per-ASN catches it — what is an ASN?) Connect to Chapter 17's "compound limits" concept.

(c) The CDN's Kafka-based purge pipeline (Chapter 15) processes purge events in partition order by cache_key_hash. A customer issues two purges for the same URL in quick succession:

• Purge 1 at T=0: expire cached object, TTL=0
• Purge 2 at T=1s: expire cached object, TTL=0 (user accidentally sent twice) Both purge events land in the same Kafka partition (same cache_key_hash). A PoP's consumer processes them in order. Is this safe? What if the consumer processes them out of order (is this possible in Kafka)? What if the customer's application sends the purges to different Kafka producers — could they land out of order in the same partition? Connect to Chapter 15's ordering guarantees (within-partition total order) and Chapter 14's at-least-once delivery (purge events may be delivered more than once — is this safe?).

What's Coming Next

Chapter 23 designs a payment processing system — the hardest correctness problem in software engineering. Everything in the curriculum has prepared you: WAL and durability (Chapter 2) for the ledger, distributed transactions (Chapter 13) for money movement across accounts, idempotency (Chapter 14) for safe retries, consensus (Chapter 12) for exactly-once execution, and observability (Chapter 18) for audit trails. The key constraint: a payment system where money moves incorrectly — even once — is a catastrophic failure. The entire design is oriented around making it physically impossible to create or destroy money through any failure scenario. It is the chapter where correctness trumps every other consideration.

Chapter 22 complete. The CDN chapter is unique in the curriculum because the bottleneck is physics — the speed of light — not software. Every architectural decision (anycast, PoP placement, cache hierarchy, origin shield) is a direct response to the fundamental constraint that electrons cannot travel faster than 200,000 km/s. Problem 4 (Zipf distribution + long tail analysis) is the most mathematically intensive problem in the curriculum — the arithmetic is straightforward, but the implications for cache design are non-obvious. Problem 5c (Kafka ordering + purge idempotency) is a subtle correctness problem that appears in distributed systems design reviews at senior engineering level.