Chapter 8: Performance — Caching & Compression

"The fastest request is the one you never make.
The smallest response is the one you never send."

1. Physical Reality First

Your API server has a finite capacity. At some point, more requests per second means more servers, more cost, more complexity. But many of those requests don't need to reach your server at all.

Consider GET /products/42 — a product detail page hit 50,000 times per hour. The product changes once a week. You're running your database query, your business logic, your serialization, and your network round-trip 50,000 times per hour for data that barely changes.

HTTP has two built-in mechanisms to eliminate this waste:

Caching: Store the response somewhere closer to the client. Serve it from there. The origin server never sees the request.

Compression: Make the response bytes smaller before sending. Less data over the wire, faster delivery.

Both are controlled entirely by HTTP headers. No application code changes. Just the right headers.

This chapter teaches you the exact headers, the exact semantics, and exactly when each directive applies — so you can eliminate 80–90% of your origin server load with configuration, not code.

2. Where Caches Live

Before the directives, understand the players. A request from a mobile client to your API can pass through multiple caches:

Mobile Client
    │
    ▼
Browser / App Cache         ← In-memory cache in the HTTP client library
    │
    ▼
CDN Edge Node               ← Cloudflare, Fastly, CloudFront (closest to user)
    │
    ▼
Reverse Proxy / API Gateway ← Nginx, Varnish, Kong (closest to your servers)
    │
    ▼
Origin Server               ← Your actual API
    │
    ▼
Database

Each layer can cache. Your Cache-Control headers tell each layer how to behave. Get them right and most requests never reach your origin. Get them wrong and you either serve stale data or bypass caching entirely.

Cache taxonomy by audience:

• Private cache: Only the end client (browser, mobile app). User-specific data.
• Shared cache: CDN or reverse proxy — serves many users. Only for public/non-user-specific data.

3. Cache-Control — The Master Header

Cache-Control is a response header (and optionally a request header) that controls caching behavior. It takes comma-separated directives.

3.1 The Core Directives

max-age=N — Cache this response for N seconds. After N seconds, the cache must revalidate.

Cache-Control: max-age=3600          ← cache for 1 hour
Cache-Control: max-age=86400         ← cache for 1 day
Cache-Control: max-age=31536000      ← cache for 1 year (static assets)

public — Any cache (CDN, proxy, browser) may store this response.

Cache-Control: public, max-age=3600

Use for: product listings, blog posts, public user profiles, any data not specific to the requester.

private — Only the end-client (browser, mobile app) may cache. CDNs must not store it.

Cache-Control: private, max-age=600

Use for: user dashboards, account pages, any data containing user-specific information.

no-cache — Cache the response, but revalidate before using it. The cache stores it but must check with the origin on every use. (Confusingly named — it does cache, it just always checks.)

Cache-Control: no-cache

Use for: data that changes frequently but conditional requests (ETags) can save bandwidth.

no-store — Do not cache at all. Not in memory, not on disk. Never.

Cache-Control: no-store

Use for: financial transactions, medical data, any response that must never be stored anywhere.

must-revalidate — Once max-age expires, the cache must revalidate before serving. It must not serve stale content even if the origin is down.

Cache-Control: max-age=3600, must-revalidate

stale-while-revalidate=N — Serve the stale cached response immediately, revalidate in the background. Client gets instant response. Next request gets the fresh one.

Cache-Control: max-age=300, stale-while-revalidate=60

After 300 seconds, the cached response is stale. For the next 60 seconds: serve stale immediately, revalidate in the background. After 360 seconds total: must revalidate before serving.

This is excellent for APIs where slight staleness is acceptable and latency matters more than perfect freshness.

s-maxage=N — Like max-age, but only for shared caches (CDNs). Overrides max-age for CDNs while leaving browser cache behavior at max-age.

Cache-Control: max-age=60, s-maxage=3600

Browser caches for 60 seconds (user sees semi-fresh data). CDN caches for 1 hour (you protect origin). After 60 seconds browser revalidates via CDN which still has it cached.

3.2 Common Cache-Control Recipes

# Public, rarely changing data (product catalog, blog posts)
Cache-Control: public, max-age=3600, stale-while-revalidate=86400

# User-specific data (profile, settings)
Cache-Control: private, max-age=300

# Real-time data (order status, live scores)
Cache-Control: no-cache

# Sensitive financial / auth data
Cache-Control: no-store, no-cache

# Static versioned assets (CSS, JS with hash in filename)
Cache-Control: public, max-age=31536000, immutable

# API response that CDN should cache longer than browser
Cache-Control: public, max-age=60, s-maxage=3600

4. ETags — Conditional Requests

Cache-Control: max-age=3600 works well when you know the staleness window. But what if you don't know how often data changes — sometimes every minute, sometimes not for a week?

ETags solve this with a different approach: don't re-send data that hasn't changed.

4.1 How ETags Work

Step 1: Server includes ETag in response

GET /products/42
→ 200 OK
   ETag: "abc123def456"
   Cache-Control: no-cache
   Content-Type: application/json

   {"id": 42, "name": "Laptop", "price": 85000, ...}

The ETag is a fingerprint of the response — typically a hash of the content or a version number.

Step 2: Client stores the ETag alongside the response

Step 3: On next request, client sends the ETag back

GET /products/42
If-None-Match: "abc123def456"

Step 4: Server checks if the resource has changed

current_etag = compute_etag(product_42)

if request.headers.get("If-None-Match") == current_etag:
    return Response(status=304)  # Not Modified — no body sent
else:
    return Response(body=product_42, headers={"ETag": current_etag})

Step 5a: If unchanged → 304 Not Modified (no body)

→ 304 Not Modified
   ETag: "abc123def456"
   
   [empty body — client uses cached copy]

The client gets a response in ~10ms with ~200 bytes (just headers). No body transmitted. No DB query needed if you cache the ETag. This is the cheapest possible API response.

Step 5b: If changed → 200 OK with new data

→ 200 OK
   ETag: "xyz789new"
   
   {"id": 42, "name": "Laptop Pro", "price": 92000, ...}

4.2 Generating ETags

Three strategies:

Hash of content (strongest):

import hashlib, json
def compute_etag(resource):
    content = json.dumps(resource, sort_keys=True)
    return hashlib.md5(content.encode()).hexdigest()
# → "abc123def456"

Accurate — changes whenever any field changes. Expensive if resource is large.

Version/timestamp from DB (practical):

def compute_etag(resource):
    return f"{resource['id']}-{resource['updated_at'].timestamp()}"
# → "42-1705312200.0"

Cheap — just a string concatenation. As long as updated_at is updated on every write, this is accurate.

Weak ETags (content may be semantically equivalent):

ETag: W/"abc123"

Prefixed with W/. Signals that two responses are semantically equivalent even if not byte-identical (e.g., whitespace differences). Less common.

4.3 Last-Modified / If-Modified-Since

An older, simpler alternative to ETags using timestamps:

GET /products/42
→ 200 OK
   Last-Modified: Mon, 15 Jan 2024 10:30:00 GMT

GET /products/42
If-Modified-Since: Mon, 15 Jan 2024 10:30:00 GMT
→ 304 Not Modified   (if unchanged)
→ 200 OK             (if changed)

ETag vs Last-Modified:

Use ETags. Use Last-Modified only for static files where modification timestamps are natural.

5. The Vary Header

When you cache a response, you cache it keyed by URL. But sometimes the same URL returns different content based on request headers — Accept-Language, Accept-Encoding, Authorization.

Without Vary, a CDN might:

1. Cache GET /products for User-Agent: Mozilla (returns English)
2. Serve that cached English response to a user with Accept-Language: hi (wants Hindi)

Vary tells the cache which request headers to include in the cache key:

Vary: Accept-Language

Now the cache stores separate copies keyed by URL + Accept-Language. English and Hindi responses are cached independently.

Common Vary uses:

Vary: Accept-Encoding    ← Different cached copies for gzip vs brotli vs uncompressed
Vary: Accept-Language    ← Different cached copies per language
Vary: Accept             ← Different cached copies per content type (JSON vs XML)
Vary: Authorization      ← Different cached copies per user (AVOID — defeats shared caching)

Never Vary: Authorization on a shared cache. You'd cache a different copy per user, per URL. That's millions of cache entries. The shared cache fills with user-specific responses and evicts everything useful.

For user-specific data: use Cache-Control: private and let the browser/app cache handle it.

6. Cache Invalidation

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

The problem: you cached GET /products/42 for 1 hour. The product's price changes at minute 10. The cache will serve the wrong price for 50 more minutes.

6.1 TTL-Based (Accept Some Staleness)

Set max-age based on acceptable staleness:

• Product descriptions: 1 hour is fine
• Product prices: maybe 5 minutes
• Order status: no-cache (user expects real-time)

Simple. No invalidation logic needed. Tradeoff: staleness window.

6.2 Cache Busting via URL

When the resource changes, change its URL:

/assets/app.js?v=a1b2c3d4     ← hash of file content
/assets/app.a1b2c3d4.js       ← hash in filename

Used for static assets, not API responses (you can't change the URL of /products/42).

6.3 Active Invalidation via CDN API

When product 42's price changes, your backend calls the CDN's purge API:

# Cloudflare purge example
def on_product_updated(product_id):
    cloudflare.purge_cache(urls=[
        f"https://api.example.com/v1/products/{product_id}",
        f"https://api.example.com/v1/products"  # also bust the list
    ])

Cloudflare, Fastly, and CloudFront all have purge APIs. This gives you fine-grained control: purge exactly the affected URLs the moment data changes.

Purge cascade: When you update product 42, you might need to purge:

• /products/42 (the product itself)
• /products (the list, which might include product 42's name/price)
• /categories/electronics/products (category listing)
• /search?q=laptop (search results containing this product)

Cache invalidation is a graph problem. Plan your invalidation strategy alongside your caching strategy.

7. Compression

HTTP responses can be compressed before sending. The savings are dramatic for JSON:

JSON compresses extremely well (70–85% reduction) because it's highly repetitive — field names repeat on every object in a list.

7.1 The Negotiation Flow

Client sends:
  Accept-Encoding: gzip, deflate, br

Server picks best supported encoding, compresses, responds:
  Content-Encoding: gzip
  [gzip-compressed body]

Client decompresses automatically.

If client sends no Accept-Encoding, server responds uncompressed.

7.2 Gzip vs Brotli

Support both. If client sends Accept-Encoding: gzip, br, prefer Brotli. If only gzip, use gzip. Most modern HTTP frameworks handle this automatically.

7.3 When NOT to Compress

• Already-compressed data: JPEG, PNG, MP4, ZIP, PDF. Compressing compressed data wastes CPU and sometimes makes it larger.
• Very small responses: Below ~1KB, compression overhead (headers + algorithm overhead) can exceed the savings.
• Streaming responses: Compression buffers data. For Server-Sent Events or chunked streaming where latency matters, don't compress.

# Skip compression for small responses
if len(response_body) < 1024:
    return response_body  # uncompressed

# Skip for already-compressed content types
if content_type in ('image/jpeg', 'image/png', 'application/zip'):
    return response_body  # uncompressed

7.4 Compression and Caching Together

When you use both compression and caching, the cache must store separate copies per encoding:

Cache key: URL + Accept-Encoding
Vary: Accept-Encoding     ← Tell the cache to vary by encoding

Without Vary: Accept-Encoding, a CDN might cache the gzip-compressed version and serve it to a client that only accepts plain text — sending garbage.

Always set Vary: Accept-Encoding when your API compresses responses.

8. Other Performance Headers

8.1 Keep-Alive and Connection Reuse

From Chapter 1: TCP handshakes are expensive. HTTP/1.1 keeps connections alive by default:

Connection: keep-alive       ← default in HTTP/1.1
Keep-Alive: timeout=60, max=100  ← close after 60s idle or 100 requests

HTTP/2 goes further — one connection, multiple concurrent streams. No explicit keep-alive needed.

For REST APIs: ensure your reverse proxy (Nginx) and app server (Gunicorn, Node) support HTTP/2 and persistent connections to each other. This eliminates repeated TCP overhead between your internal services.

8.2 Prefetch and Link Hints

Link: </products/43>; rel="prefetch"

Tells the client that it'll probably need product 43 next. Modern browsers act on this and fetch it in advance. Useful for paginated responses where the next page URL is predictable.

8.3 Response Time Headers

X-Response-Time: 23ms          ← time spent in your application code
Server-Timing: db;dur=12, app;dur=8, total;dur=23  ← detailed breakdown

Server-Timing (W3C standard) lets you expose internal timing to the browser's DevTools. Invaluable for debugging slow endpoints. Not a caching mechanism — a performance observability mechanism.

9. Caching Strategy by Resource Type

10. How Real Systems Cache

GitHub:

Cache-Control: private, max-age=60, s-maxage=60
ETag: W/"abc123"

Private (user-specific), 60-second max-age, ETag for conditional requests. Authenticated API responses are private.

For public repo data (unauthenticated):

Cache-Control: public, max-age=60, s-maxage=60

Stripe: Stripe's API responses are not cacheable (financial data must be fresh):

Cache-Control: no-cache, no-store

Stripe's static assets (JS libraries, documentation) use long-lived caching with versioned URLs.

Cloudflare (CDN behavior): Cloudflare respects Cache-Control headers by default. When your API sets public, max-age=3600, Cloudflare caches it at 200+ edge locations globally. A user in Chennai hits the Cloudflare PoP in Chennai — not your origin server in US-East.

Cloudflare also supports Cache Rules — override Cache-Control from the CDN side without touching your API code. Useful when you inherit an API with no caching headers.

Twitter/X: Timeline responses use aggressive edge caching with short TTLs + stale-while-revalidate:

Cache-Control: public, max-age=5, stale-while-revalidate=55

Serve from cache immediately (5s fresh). Background revalidate for the next 55 seconds. Users get instant response; cache refreshes every ~5-60 seconds depending on traffic.

11. Vocabulary Map

12. Practice Problems

Problem 1 — Failure Analysis: Your API returns Cache-Control: public, max-age=3600 for GET /users/42/profile. User 42 updates their profile picture. For the next hour, all users see the old picture from the CDN cache. Identify two specific fixes — one that solves it immediately (without waiting for TTL), and one that prevents this class of bug architecturally. Explain the trade-off between these approaches.

Problem 2 — Design From Scratch: Design the complete caching strategy for an e-commerce API with these endpoints: GET /products/{id} (product detail), GET /products (product listing), GET /cart (current user's cart), GET /orders/{id} (order detail), GET /users/me (current user). For each: specify the Cache-Control header, whether to use ETags, whether a CDN should cache it, and the invalidation strategy when data changes.

Problem 3 — Synthesis: Your API serves 10 million requests/day. A performance audit shows: 70% are GET /products/{id} with an average DB query time of 8ms, and the product catalog (50,000 products) changes about 200 times/day. Calculate: (a) how many DB queries/day you eliminate with Cache-Control: public, max-age=3600 and a hit rate of 85%, (b) with ETags and no-cache at a 90% 304 rate, (c) with stale-while-revalidate=3600 at 95% immediate serve rate. Which approach gives the best balance of freshness and origin protection?

Problem 4 — Go Deeper: Implement the complete ETag flow for GET /orders/{id} in pseudocode. The order has a version integer that increments on every update. Show: (a) generating and returning the ETag on the first request, (b) handling the conditional request (If-None-Match header present), (c) what happens if the client sends an ETag from an order that no longer exists (404), (d) what Cache-Control header accompanies the ETag and why. Also explain the difference between a strong ETag and a weak ETag and when you'd use each.

Problem 5 — Connect to Previous: From Chapter 7, cursor-based pagination returns responses like GET /orders?after=cursor_abc. From this chapter, you know Vary: Accept-Encoding is required when compressing. You also know CDNs cache by URL. A client requests GET /orders?after=cursor_abc with Accept-Encoding: gzip and another client requests the same URL with Accept-Encoding: br. Trace through: (a) what the CDN stores without Vary, (b) what it serves to the second client (wrong), (c) what Vary headers fix this, (d) whether cursor-paginated responses should be cached at all, and why the answer is nuanced.

What's Coming Next

Your API is fast. Now: is it safe?

Chapter 9 covers API security — not auth (that was Chapter 5) but the full attack surface: the OWASP API Top 10, injection attacks, mass assignment, CORS, input validation, HTTPS enforcement, and security headers. Security is not a feature you add later. Every chapter so far has had security implications — Chapter 9 assembles them into a complete threat model.