Chapter 1: HTTP — The Wire Your API Runs On

"Before you design an API, you must understand the protocol it rides on.
REST is not magic — it is HTTP used deliberately."

1. Physical Reality First

Your API does not live in the cloud. It lives on a machine with a CPU, RAM, and a network card.
When a client calls your API, here is what physically happens:

The client's OS opens a TCP socket to your server's IP address on port 443 (HTTPS) or 80 (HTTP)
A TCP handshake completes: SYN → SYN-ACK → ACK (1.5 round trips, ~100ms across continents)
For HTTPS, a TLS handshake follows: certificates exchanged, keys negotiated (~1–2 more round trips)
Now the client sends bytes down this socket — structured text that follows the HTTP specification
Your server reads those bytes, parses them, runs your code, and writes bytes back
The client reads those bytes and parses them

HTTP is just a text protocol on top of TCP. That's it. No magic. Just structured bytes.

Here is what those bytes look like:

code

GET /users/42 HTTP/1.1\r\n
Host: api.example.com\r\n
Authorization: Bearer eyJhbGci...\r\n
Accept: application/json\r\n
\r\n

And the response:

code

HTTP/1.1 200 OK\r\n
Content-Type: application/json\r\n
Content-Length: 87\r\n
\r\n
{"id": 42, "name": "Trilochan", "email": "t@example.com", "created_at": "2024-01-15"}

The \r\n is a carriage return + newline. A blank \r\n line separates headers from body.
That separator is all that exists between metadata and data. This is not an abstraction — it is a literal byte sequence.

Latency reality:

Same datacenter: ~1ms
Same city: ~5ms
Cross-continent: ~100–200ms
Mobile on 4G: ~50–150ms + TCP + TLS setup

TCP + TLS setup can cost 300–500ms on the first request. This is why persistent connections exist.

2. The HTTP Request — Dissected

Every HTTP request has exactly three parts:

code

[METHOD] [PATH] [HTTP-VERSION]\r\n      ← Request Line
[Header-Name]: [Header-Value]\r\n       ← Headers (zero or more)
\r\n                                     ← Blank line (mandatory separator)
[Body]                                   ← Optional body

2.1 The Method (Verb)

The method tells the server what action to perform. There are 9 official HTTP methods but only 5 matter for APIs:

Method	Meaning	Has Body?	Safe?	Idempotent?
GET	Retrieve a resource	No	Yes	Yes
POST	Create a new resource	Yes	No	No
PUT	Replace a resource entirely	Yes	No	Yes
PATCH	Partially update a resource	Yes	No	No*
DELETE	Remove a resource	No	No	Yes

Safe = causes no side effects (read-only).
Idempotent = calling it N times has the same effect as calling it once.

Why does idempotency matter? Network failures. When a DELETE times out, can you safely retry? Yes — deleting an already-deleted resource is fine. When a POST times out, retrying might create a duplicate order. This is why POST requires idempotency keys (covered in Chapter 5).

*PATCH idempotency depends on implementation. Set name to "Alice" is idempotent. Increment counter by 1 is not.

2.2 The Path

code

/users/42/orders?status=pending&page=2
  ↑           ↑         ↑
  resource  identifier  query params

The path has two parts:

Path (/users/42/orders): identifies the resource
Query string (?status=pending&page=2): modifies the request (filter, sort, paginate)

Query params are not encrypted even over HTTPS — they appear in server logs, browser history, and CDN logs. Never put secrets or passwords in query params.

2.3 Headers

Headers are key-value metadata about the request. The most important ones:

code

Host: api.example.com           ← Required. Which server to route to (virtual hosting)
Authorization: Bearer <token>   ← Auth credential
Content-Type: application/json  ← Format of the request body
Accept: application/json        ← Format the client wants in the response
Content-Length: 47              ← Bytes in the body
User-Agent: MyApp/2.1           ← Client identification

Host is mandatory in HTTP/1.1. Without it, the server doesn't know which of potentially many virtual-hosted apps you're talking to.

2.4 The Body

Only some methods carry a body (POST, PUT, PATCH). The body is raw bytes.
The Content-Type header tells the server how to parse those bytes:

Content-Type	When Used
`application/json`	JSON — use this for REST APIs
`application/x-www-form-urlencoded`	HTML form data (`key=value&key2=value2`)
`multipart/form-data`	File uploads
`application/octet-stream`	Raw binary

3. The HTTP Response — Dissected

code

[HTTP-VERSION] [STATUS-CODE] [REASON-PHRASE]\r\n   ← Status Line
[Header-Name]: [Header-Value]\r\n                   ← Response Headers
\r\n                                                 ← Separator
[Body]                                               ← Response Body

3.1 Status Codes

Status codes are 3-digit integers. The first digit tells you the class:

Class	Range	Meaning
1xx	100–199	Informational (rare in APIs)
2xx	200–299	Success
3xx	300–399	Redirect
4xx	400–499	Client error (you sent something wrong)
5xx	500–599	Server error (we broke something)

The codes that matter most for REST APIs:

code

200 OK              — Request succeeded, body contains result
201 Created         — POST succeeded, new resource created
204 No Content      — Success but no body (common for DELETE)
400 Bad Request     — Malformed request, invalid JSON, missing field
401 Unauthorized    — Not authenticated (no token, or token invalid)
403 Forbidden       — Authenticated but not authorized (you can't access this)
404 Not Found       — Resource does not exist
409 Conflict        — State conflict (duplicate create, version mismatch)
422 Unprocessable   — Body parsed fine but semantically invalid (email format wrong)
429 Too Many Req    — Rate limited
500 Internal Error  — Server bug
503 Unavailable     — Server temporarily down (use for circuit breaking)

Critical distinction: 401 vs 403.

401: "Who are you?" — no valid credentials
403: "I know who you are, but you can't do this" — valid credentials, insufficient permissions

Most APIs get this wrong and return 403 for both. Don't.

3.2 Response Headers

code

Content-Type: application/json; charset=utf-8   ← Body format
Content-Length: 312                              ← Body size in bytes
Cache-Control: max-age=3600                      ← Caching directive
ETag: "abc123"                                   ← Resource version fingerprint
Location: /users/42                              ← Where the new resource lives (after 201)
X-Request-ID: req_8f2a3b                         ← Tracing ID
Retry-After: 60                                  ← Seconds to wait (after 429 or 503)

4. HTTP Versions — What Changed and Why

HTTP/1.0 (1996): One request per connection

Each request opened a new TCP connection. Expensive. A page with 30 assets = 30 TCP handshakes.

HTTP/1.1 (1997): Persistent connections

The Connection: keep-alive header (default in 1.1) keeps the TCP connection open for multiple requests.
Problem: Head-of-line blocking — requests must complete in order. Request 3 can't be sent until Request 2's response arrives.

HTTP/2 (2015): Multiplexing

Multiple requests fly in parallel over a single TCP connection using binary framing and streams.
No head-of-line blocking at the HTTP layer. Also adds header compression (HPACK) — headers repeat a lot; compressing them saves bandwidth.

HTTP/3 (2022): QUIC replaces TCP

Built on UDP with connection state in userspace. Eliminates TCP head-of-line blocking entirely (TCP retransmits block the entire stream). Faster connection setup (0-RTT on reconnect).

What this means for your API:

HTTP/2 is the baseline you should support today
HTTP/3 is a CDN/load balancer concern — your app server doesn't need to handle it directly
For REST APIs, these differences matter for latency, not semantics. Your API design is the same across all versions.

5. The Request Lifecycle (Full Picture)

When POST /orders hits your backend, here is the complete journey:

code

Client
  │
  ├─ DNS lookup: api.example.com → 104.18.20.1        ~5ms (cached: 0ms)
  ├─ TCP handshake: SYN/SYN-ACK/ACK                   ~50ms
  ├─ TLS handshake: ClientHello/ServerHello/...        ~100ms
  │
  ├─ HTTP request bytes sent
  │
Load Balancer / Reverse Proxy (Nginx, Envoy)
  ├─ TLS termination
  ├─ Route to upstream app server
  │
App Server
  ├─ HTTP framework parses bytes → Request object
  ├─ Middleware runs: auth, rate limiting, logging
  ├─ Route handler executes
  ├─ Database queries
  ├─ Business logic
  ├─ Serialize response → bytes
  │
  └─ Response bytes travel back to client

Total time for that POST /orders: probably 50–200ms in production.
Of that, your code might take 5ms. The network and database take the rest.

6. Content Negotiation

HTTP has a built-in mechanism for clients to say what format they want:

code

Accept: application/json, application/xml;q=0.8, */*;q=0.5

The q value is preference (0–1). The client prefers JSON, will accept XML, will accept anything else.

The server responds with:

code

Content-Type: application/json

Or, if it can't satisfy any preference:

code

406 Not Acceptable

In practice, most APIs only speak JSON and ignore Accept. But knowing this exists matters when you build APIs that serve multiple formats (v1 returns XML, mobile client wants JSON, webhook needs a specific schema).

7. How Real Systems Use This

GitHub API:

Uses Link response header for pagination instead of body envelopes
ETag + If-None-Match for conditional requests (304 Not Modified saves bandwidth)
X-RateLimit-Remaining / X-RateLimit-Reset in every response

Stripe API:

Idempotency-Key request header — POST to /charges with the same key twice = one charge
Stripe-Version header to pin API version per request
Always returns request_id in response for support/debugging

Twilio:

Uses HTTP Basic Auth with Account SID as username
Webhooks are POST requests Twilio makes to your server
Signature in X-Twilio-Signature header for webhook verification

AWS API Gateway:

Parses Authorization: AWS4-HMAC-SHA256 — a signed header scheme
Rate limiting via x-amzn-RequestId tracked per API key
OPTIONS preflight requests handled for CORS

8. Vocabulary Map

Term	Definition
TCP	Transport layer protocol. Guarantees delivery and ordering of bytes. HTTP runs on top of it.
TLS	Transport Layer Security. Encrypts the TCP connection. HTTPS = HTTP over TLS.
Request Line	First line of an HTTP request: METHOD PATH HTTP-VERSION
Status Line	First line of an HTTP response: HTTP-VERSION STATUS-CODE REASON
Header	Key-value metadata sent with a request or response. Separated from body by blank line.
Body	Raw bytes carrying the payload (JSON, form data, binary). Optional.
Idempotent	Operation that produces the same result if applied once or N times.
Safe	Method that causes no side effects (read-only). GET is safe; POST is not.
Content-Type	Header declaring the format of the body being sent.
Accept	Header declaring what format the client wants in the response.
Persistent Connection	TCP connection reused across multiple HTTP requests (HTTP/1.1 default).
Multiplexing	HTTP/2 feature: multiple requests in-flight simultaneously on one TCP connection.
Head-of-Line Blocking	HTTP/1.1 problem: request N waits for request N-1 to complete.
Content Negotiation	Protocol for client and server to agree on response format via Accept/Content-Type.
Round Trip	One request + one response. A TCP handshake costs 1.5 round trips.

9. Practice Problems

Problem 1 — Failure Analysis:
A client reports that every first request to your API takes 500ms but subsequent requests take 20ms. The API is returning correct data both times. What is the cause? How would you fix it?

Problem 2 — Design From Scratch:
You are building an API endpoint that allows users to upload a profile picture (JPEG, max 5MB) and update their display name simultaneously. Design the full HTTP request — method, path, headers, and body format. Justify every choice.

Problem 3 — Synthesis:
A client calls DELETE /orders/99 and the TCP connection drops before the response arrives. The client retries. What are the possible outcomes? What must your server do to make this safe? Use the concept of idempotency in your answer.

Problem 4 — Go Deeper:
Explain what happens when a browser makes a cross-origin API request (e.g., https://app.example.com calling https://api.example.com). What HTTP mechanism is involved? What request does the browser send before the actual request, and why? What must your API return?

Problem 5 — Connect to Previous:
You learned in the Distributed Systems book (Chapter 9) that caches can serve stale data. HTTP has a built-in mechanism to validate whether a cached response is still fresh without downloading the full body. Name the two HTTP headers involved, describe the round trip they enable, and explain what status code is returned when the cached version is still valid.

What's Coming Next

You now understand the protocol. Every REST API is HTTP used with specific conventions.

Chapter 2 explains those conventions — the six REST constraints that Fielding defined in his PhD thesis, why they exist, and how they change the way you design URLs, methods, and state. We will also look at the Richardson Maturity Model: a rubric for measuring how RESTful your API actually is.

The jump from "HTTP" to "REST" is the jump from protocol to architecture. That's next.