Chapter 6: Error Handling

"Success responses are for users. Error responses are for developers.
A good error tells you exactly what went wrong, why, and how to fix it."

1. Physical Reality First

When your API fails, a developer reads the response — at 2am, during an incident, under pressure. They need to know three things instantly:

What went wrong
Why it went wrong
How to fix it

If your error response is this:

json

{"error": "Something went wrong"}

The developer has none of those three things. They have to guess, check logs, reproduce the error, and waste 30 minutes on something that should have taken 30 seconds.

If your error response is this:

json

{
  "type": "https://api.example.com/errors/validation-error",
  "title": "Validation Failed",
  "status": 422,
  "detail": "The request body contains invalid fields.",
  "instance": "/orders",
  "request_id": "req_8f2a3b4c",
  "errors": [
    {
      "field": "email",
      "code": "invalid_format",
      "message": "Must be a valid email address. Got: 'trilochan@'"
    },
    {
      "field": "amount",
      "code": "out_of_range",
      "message": "Must be between 1 and 1000000 (in paise). Got: -500"
    }
  ]
}

The developer fixes the bug in 30 seconds.

Error design is not cosmetic. It directly determines how fast your API consumers can build, debug, and ship. This chapter builds a complete, production-grade error handling system.

2. The Status Code Is Not Enough

Status codes tell you the class of error. They don't tell you the cause.

Both of these are 400:

Missing required field email
JSON body is malformed (unclosed bracket)

Both of these are 422:

Email field has invalid format
Credit card number fails Luhn check

Both of these are 409:

Duplicate create (same idempotency key, different payload)
Optimistic locking conflict (resource was modified since you read it)

Clients need to distinguish between these to handle them correctly. A client can auto-fix a missing field by prompting the user. It cannot auto-fix a Luhn check failure the same way. The status code alone is insufficient — you need machine-readable error codes in the body.

3. The RFC 7807 Standard: Problem Details

RFC 7807 (2016) defines a standard JSON structure for error responses. Use it.

code

Content-Type: application/problem+json

json

{
  "type":     "https://api.example.com/errors/insufficient-funds",
  "title":    "Insufficient Funds",
  "status":   402,
  "detail":   "Your account balance (₹500) is less than the charge amount (₹2500).",
  "instance": "/payments/attempt-77"
}

The five standard fields:

Field	Type	Purpose
`type`	URI (string)	Machine-readable error type. A URL that docs the error.
`title`	string	Short human-readable summary. Same for all instances of this error type.
`status`	integer	The HTTP status code (repeated in body for convenience).
`detail`	string	Human-readable explanation specific to this occurrence.
`instance`	URI (string)	URI that identifies this specific error occurrence (optional).

Why type is a URI: It's globally unique (no namespace collisions) and should ideally resolve to documentation for that error type. Even if the URL doesn't resolve, it uniquely identifies the error.

You can extend the standard with additional fields — RFC 7807 explicitly allows it:

json

{
  "type": "https://api.example.com/errors/validation-error",
  "title": "Validation Failed",
  "status": 422,
  "detail": "The request body contains 2 invalid fields.",
  "request_id": "req_8f2a3b4c",
  "errors": [...]
}

4. Machine-Readable Error Codes

The type URI handles high-level error categories. For finer grain, use a code field:

json

{
  "type": "https://api.example.com/errors/validation-error",
  "title": "Validation Failed",
  "status": 422,
  "code": "email_invalid_format",
  "detail": "The email field must be a valid email address."
}

Error code design rules:

snake_case, all lowercase: card_declined, insufficient_funds, rate_limit_exceeded
Specific, not generic: email_invalid_format not validation_error
Stable: Never rename a code — clients write if (error.code === 'card_declined') and shipping code breaks
Document all codes: Every possible error code in your API should be in your docs with: what caused it, how to fix it

Code taxonomy example for a payments API:

code

authentication_required     → no auth header
invalid_api_key             → auth header present but key not found
api_key_expired             → key was revoked
insufficient_permissions    → key lacks required scope

card_declined               → card issuer declined
card_expired                → card expiry date is past
card_insufficient_funds     → not enough balance
card_invalid_number         → Luhn check failed
card_lost_or_stolen         → issuer flagged card

rate_limit_exceeded         → too many requests
quota_exceeded              → monthly quota hit

resource_not_found          → /orders/99 doesn't exist
resource_already_exists     → duplicate create
resource_conflict           → optimistic lock conflict

validation_error            → generic validation failure (use specific sub-codes)

5. Validation Errors: Multiple Fields at Once

The worst UX: submit a form with 5 errors, fix the first one, resubmit, see the second error, repeat 5 times.

Your API should return all validation errors at once:

json

POST /users
{
  "name": "",
  "email": "not-an-email",
  "age": -5,
  "password": "abc"
}

→ 422 Unprocessable Entity
{
  "type": "https://api.example.com/errors/validation-error",
  "title": "Validation Failed",
  "status": 422,
  "detail": "The request contains 4 validation errors.",
  "request_id": "req_8f2a3b4c",
  "errors": [
    {
      "field": "name",
      "code": "required",
      "message": "Name is required and cannot be empty."
    },
    {
      "field": "email",
      "code": "invalid_format",
      "message": "Must be a valid email address."
    },
    {
      "field": "age",
      "code": "out_of_range",
      "message": "Age must be between 0 and 150. Got: -5"
    },
    {
      "field": "password",
      "code": "too_short",
      "message": "Password must be at least 8 characters. Got: 3 characters."
    }
  ]
}

The errors array fields:

Field	Purpose
`field`	Which field failed. Use dot notation for nested: `address.city`.
`code`	Machine-readable error type for this field.
`message`	Human-readable explanation. Include what was received vs what was expected.

For nested objects:

json

{
  "field": "address.postal_code",
  "code": "invalid_format",
  "message": "Postal code must be 6 digits. Got: 'ABC'"
}

For array items:

json

{
  "field": "line_items[2].quantity",
  "code": "out_of_range",
  "message": "Quantity must be between 1 and 999. Got: 0"
}

6. Status Code Precision — The Complete Guide

This is where most APIs are sloppy. Use the right code every time.

4xx — Client Errors (you did something wrong)

code

400 Bad Request
    Use for: malformed JSON, missing required field, wrong data type
    Not for: semantic errors (email format wrong) — that's 422

401 Unauthorized
    Use for: no auth credentials, invalid token, expired token
    Never return: a body that reveals WHETHER the user exists
    Message: "Authentication required" (not "User not found")

403 Forbidden
    Use for: valid auth, but insufficient permissions or IDOR
    Classic mistake: returning 404 instead of 403 to hide resource existence
    Security note: sometimes intentionally return 404 to prevent enumeration attacks

404 Not Found
    Use for: resource genuinely doesn't exist
    Security use: return 404 instead of 403 when you don't want to reveal existence

405 Method Not Allowed
    Use for: POST /users/42 when only GET and PATCH are supported
    Must include: Allow: GET, PATCH header showing valid methods

409 Conflict
    Use for: state conflicts — duplicate create, version mismatch
    Use for: trying to delete something that has dependents

410 Gone
    Use for: resource existed but was permanently deleted (vs 404: never existed)
    Use for: deprecated endpoints that have been removed

415 Unsupported Media Type
    Use for: client sent Content-Type: text/plain, you only accept application/json

422 Unprocessable Entity
    Use for: body parsed successfully (valid JSON) but semantically invalid
    Use for: email format invalid, amount negative, enum value not in allowed set
    Distinction from 400: 400 = can't parse; 422 = parsed but invalid

429 Too Many Requests
    Use for: rate limiting
    Must include: Retry-After header (seconds to wait)
    Must include: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset

5xx — Server Errors (we broke something)

code

500 Internal Server Error
    Use for: unhandled exceptions, bugs, unexpected states
    Never expose: stack traces, internal error messages, DB queries
    Always log: the full error server-side; return generic message to client

502 Bad Gateway
    Use for: your service is up but an upstream it depends on is down
    Example: your API is running but the database is unreachable

503 Service Unavailable
    Use for: intentional downtime, maintenance, circuit breaker open
    Must include: Retry-After header

504 Gateway Timeout
    Use for: upstream service (DB, third-party API) timed out

The 200 OK Anti-Pattern

Never do this:

json

HTTP/1.1 200 OK
{
  "success": false,
  "error": "User not found"
}

This is the worst of both worlds: status code says success, body says failure. Clients that check only the status code (common in HTTP libraries) will treat this as success. Monitoring tools that track 4xx/5xx rates won't count this error. Every layer in your stack — load balancers, CDNs, tracing tools — uses status codes to understand health. Abuse them and you blind your own infrastructure.

7. Never Expose Internal Details

json

// ❌ Dangerous — exposes infrastructure
{
  "error": "ERROR: duplicate key value violates unique constraint \"users_email_key\"",
  "query": "INSERT INTO users (email) VALUES ('t@example.com')",
  "table": "users",
  "constraint": "users_email_key"
}

// ✅ Safe — hides infrastructure, still useful
{
  "type": "https://api.example.com/errors/resource-already-exists",
  "title": "User Already Exists",
  "status": 409,
  "code": "email_already_registered",
  "detail": "An account with this email address already exists.",
  "request_id": "req_8f2a3b4c"
}

What to never expose in errors:

Stack traces
SQL queries or schema details
Internal hostnames or IP addresses
Library/framework names and versions
File paths
Internal error codes from third-party services

Log all of this server-side. Return none of it to the client.

The request_id is the bridge: client tells you req_8f2a3b4c, you look it up in your logs and see everything.

8. Error Handling for Special Cases

8.1 Authentication Errors: Don't Reveal User Existence

json

// ❌ Reveals that this email is registered
POST /auth/login {"email": "t@example.com", "password": "wrong"}
→ 401 {"error": "Invalid password"}

// ❌ Reveals that this email is NOT registered
POST /auth/login {"email": "nobody@example.com", "password": "wrong"}
→ 401 {"error": "User not found"}

// ✅ Reveals nothing — same response for both cases
→ 401 {
    "type": "https://api.example.com/errors/authentication-failed",
    "title": "Authentication Failed",
    "status": 401,
    "detail": "Invalid email or password."
  }

User enumeration — figuring out which emails are registered — helps attackers. Return identical errors for "wrong password" and "user not found."

8.2 Rate Limit Errors: Tell Clients Exactly When to Retry

code

HTTP/1.1 429 Too Many Requests
Retry-After: 47
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1705312800

{
  "type": "https://api.example.com/errors/rate-limit-exceeded",
  "title": "Rate Limit Exceeded",
  "status": 429,
  "detail": "You have exceeded 1000 requests per hour. Try again in 47 seconds.",
  "retry_after": 47
}

A client that receives a 429 with Retry-After: 47 can sleep 47 seconds and retry automatically. Without Retry-After, the client either hammers the API (worsening the problem) or uses exponential backoff (guesses).

8.3 Async Operations: Pending ≠ Error

Sometimes an operation starts but doesn't complete immediately (payment processing, report generation, email sending).

json

POST /reports/generate
→ 202 Accepted
{
  "id": "report_7f3a",
  "status": "processing",
  "check_url": "/reports/report_7f3a",
  "estimated_completion": "2024-01-15T10:35:00Z"
}

GET /reports/report_7f3a
→ 200 OK (while processing)
{
  "id": "report_7f3a",
  "status": "processing",
  "progress": 45
}

GET /reports/report_7f3a
→ 200 OK (when failed)
{
  "id": "report_7f3a",
  "status": "failed",
  "error": {
    "code": "data_source_unavailable",
    "message": "Could not connect to the analytics database."
  }
}

202 Accepted means "we received it and it's being processed." The error, if any, lives in the resource's status field — not in the HTTP status code of a subsequent GET.

9. Error Logging and Observability

Every error your API returns should produce a log entry. The log entry should include:

json

{
  "timestamp": "2024-01-15T10:30:00.123Z",
  "request_id": "req_8f2a3b4c",
  "method": "POST",
  "path": "/orders",
  "status": 422,
  "error_code": "validation_error",
  "user_id": 42,
  "duration_ms": 12,
  "errors": ["email_invalid_format", "amount_out_of_range"]
}

Log 4xx at WARN level (client's fault, but worth tracking — a spike in 400s means your client is broken).
Log 5xx at ERROR level (your fault, requires immediate attention).
Never log sensitive data — no passwords, tokens, credit card numbers in logs.

Build dashboards tracking:

Error rate by status code
Most common error codes
Error rate by endpoint
P95/P99 of error response time

10. How Real Systems Handle Errors

Stripe:

json

{
  "error": {
    "code": "card_declined",
    "decline_code": "insufficient_funds",
    "message": "Your card has insufficient funds.",
    "param": null,
    "type": "card_error",
    "charge": "ch_3OaB1234"
  }
}

Note: Stripe uses type at a high level (card_error, api_error, validation_error) and code for specifics. The decline_code gives the bank's reason. param names the invalid field for validation errors.

GitHub:

json

{
  "message": "Validation Failed",
  "errors": [
    {
      "resource": "Issue",
      "field": "title",
      "code": "missing_field"
    }
  ],
  "documentation_url": "https://docs.github.com/rest/issues"
}

documentation_url directly links to the relevant docs page. Simple, useful.

Twilio:

json

{
  "code": 21211,
  "message": "The 'To' number +15551234567 is not a valid phone number.",
  "more_info": "https://www.twilio.com/docs/errors/21211",
  "status": 400
}

Integer error code (not string) — a Twilio-specific choice. more_info is a direct link to error documentation.

AWS:

xml

<ErrorResponse>
  <Error>
    <Type>Sender</Type>
    <Code>InvalidParameterValue</Code>
    <Message>Value (sender@example.com) for parameter From is invalid. Reason: Not verified</Message>
  </Error>
  <RequestId>1234-5678</RequestId>
</ErrorResponse>

AWS uses XML (legacy). Type is Sender (client's fault) or Receiver (AWS's fault). This maps to 4xx vs 5xx.

11. Vocabulary Map

Term	Definition
RFC 7807	IETF standard defining the `application/problem+json` format for API error responses.
Problem Details	The RFC 7807 error format: `type`, `title`, `status`, `detail`, `instance`.
Error code	Machine-readable string identifying the specific error type. Stable across API versions.
Error message	Human-readable description. Can change; clients must not parse it programmatically.
Validation error	422 response when request body is syntactically valid but semantically wrong.
400 vs 422	400: can't parse the body. 422: parsed fine, but the values are invalid.
401 vs 403	401: not authenticated. 403: authenticated but not authorized.
User enumeration	Security attack: detecting which usernames/emails are registered via error messages.
202 Accepted	HTTP status for async operations: request accepted, processing not complete.
Retry-After	Response header with seconds to wait before retrying (429, 503).
IDOR	Insecure Direct Object Reference: accessing a resource you don't own via its ID. (From Ch. 5)
Error rate	Percentage of requests returning 4xx or 5xx. Key API health metric.
Stack trace	Internal error details showing code execution path. Never expose to API clients.
request_id	Unique ID per request included in error responses for debugging. (From Ch. 4)
Decline code	Bank/payment processor's reason for card decline (e.g., `insufficient_funds`).

12. Practice Problems

Problem 1 — Failure Analysis: Your API returns 200 OK for all responses, with {"success": true} or {"success": false, "message": "..."} in the body. Three specific systems break as a result: (a) your load balancer's health check, (b) your CDN's cache, (c) your monitoring system's error rate alerts. For each, explain exactly why it breaks and what the impact is.

Problem 2 — Design From Scratch: Design the complete error response schema for an e-commerce order API. Define: (a) at least 8 error codes covering auth, validation, business logic, and server errors; (b) the JSON structure for a validation error with 3 field-level errors; (c) the JSON structure for a card decline error; (d) what HTTP status code each gets. Use RFC 7807 as the base.

Problem 3 — Synthesis: A user submits a password reset request: POST /auth/password-reset {"email": "t@example.com"}. Design the complete error handling for this endpoint covering: (a) the email doesn't exist, (b) the email exists — success, (c) the email exists but the account is locked, (d) the email has requested 5 resets in the last hour (rate limited). Which of these cases must return identical responses, and why? What status code does each get?

Problem 4 — Go Deeper: Your payment API processes charges asynchronously. POST /charges returns 202 Accepted immediately. The charge is processed in a background job and might fail 30 seconds later (card declined, fraud detected). Design the complete pattern: polling endpoint, webhook notification, and the error structure for the delayed failure. What is the status field state machine for a charge resource?

Problem 5 — Connect to Previous: From Chapter 5, you know refresh tokens can be stolen. Design the error response chain for this attack scenario: (1) attacker uses stolen refresh token to get new access token, (2) legitimate user tries to use their refresh token (now invalidated by rotation), (3) legitimate user makes API call with their old (now invalid) access token. For each step, write the exact HTTP status code, error code, and detail message. What Retry-After value should step 2 return, and why?

What's Coming Next

Your API communicates failure well. Now: how does it evolve without breaking things?

Chapter 7 covers versioning, pagination, and filtering — the three design decisions that determine whether your API can grow and serve millions of records without falling apart. Versioning is where teams make permanent, unfixable mistakes on day one. Pagination is where APIs silently corrupt data under load. We'll do both right, from the start.