Backend Engineering: First Principles

One document. No fluff. Every concept explained from the ground up — what it is, why it exists, what breaks without it, and how it connects to everything else.

How to read this

Every section answers three questions in order:

1. Why does this exist? — the problem it solves
2. What is it exactly? — the precise mental model
3. What breaks without it? — the real consequences

Read it once top to bottom. The order is deliberate — each section builds on the last.

Part 1: HTTP — The Foundation Everything Else Sits On

Before auth, middleware, REST — there is HTTP. You cannot reason about any of what follows without understanding this first.

What HTTP is

HTTP (HyperText Transfer Protocol) is a request-response protocol. A client sends a request. A server sends exactly one response. That's it. Every API you build is just HTTP — a structured way to exchange text over a network.

HTTP is stateless. The server has zero memory of previous requests. Request 1 and Request 2 are completely independent as far as the protocol is concerned. This is not a bug. It is a deliberate design that makes servers infinitely scalable — any server in a cluster can handle any request because no state is assumed.

The consequence: all state must be carried in the request itself, or stored somewhere external (database, cache, session store). This is why auth tokens exist — they are how a stateless protocol carries identity.

The anatomy of an HTTP request

Every HTTP request has exactly four parts:

METHOD /path/to/resource HTTP/1.1    ← request line
Header-Name: Header-Value            ← headers (zero or more)
                                     ← blank line (mandatory separator)
{ "body": "content" }               ← body (optional)

Method — declares intent. GET means "give me data". POST means "create something". PUT means "replace this entirely". PATCH means "update part of this". DELETE means "remove this". The method is a contract between client and server about what is going to happen.

Path — identifies the resource. /users/42 refers to user with ID 42. /users refers to the collection of users. Resources are nouns, never verbs.

Headers — metadata about the request. Not the data itself, but information about the data or the request: what format the body is in, who is making the request, what formats the client can accept, authentication tokens.

Body — the actual payload. Only present on POST, PUT, PATCH. GET and DELETE have no body — they carry all information in the URL and headers.

The anatomy of an HTTP response

HTTP/1.1 200 OK                      ← status line
Content-Type: application/json       ← headers
                                     ← blank line
{ "id": 42, "name": "Trilochan" }   ← body

Status code — the single most important thing in a response. It is a three-digit number that tells the client what happened, without them reading the body.

The ranges are not arbitrary:

• 1xx — informational (rarely used directly)
• 2xx — success (the request worked as intended)
• 3xx — redirection (go somewhere else)
• 4xx — client error (the client did something wrong)
• 5xx — server error (the server failed)

The critical principle: 4xx is the client's fault. 5xx is your fault. Never return a 500 when the client sent bad data. Never return a 400 when your database is down.

Most-used codes you must know cold:

The 401 vs 403 distinction matters deeply. 401 means "I don't know who you are — authenticate first." 403 means "I know exactly who you are — you just can't do this." Confusing these leaks information and breaks client logic.

Never return 200 with an error in the body. This is the most common amateur mistake. If something failed, the status code must reflect it. { "success": false, "error": "not found" } with a 200 is a lie to the client.

Part 2: Headers — The Metadata Layer

Headers are key-value pairs that carry metadata. They have nothing to do with the resource itself — they are information about the request or response.

Request headers your server will read

Content-Type — tells the server what format the body is in.

Content-Type: application/json
Content-Type: multipart/form-data
Content-Type: application/x-www-form-urlencoded

If a client sends Content-Type: application/json but the body is not valid JSON, reject it immediately with a 400. Do not attempt to parse it.

Accept — tells the server what format the client can handle in the response. Mostly application/json for APIs. Respect it if you support multiple formats.

Authorization — carries credentials. Almost always one of two forms:

Authorization: Bearer <jwt-token>
Authorization: Basic <base64(username:password)>

Bearer token (JWT) is the standard for APIs. Basic auth is only for server-to-server or very simple cases — never expose it to browsers without HTTPS.

X-Request-ID or X-Correlation-ID — a unique ID the client (or you, on arrival) attaches to every request. Propagated through all logs so you can trace a single request across your entire system. More on this in the Logging section.

User-Agent — identifies the client. Browser, mobile app, curl, etc. Useful for analytics and occasionally for security (rate limiting bots).

X-Forwarded-For — when your server sits behind Nginx or a load balancer, the real client IP is in this header. The req.ip in Express gives you the proxy IP, not the user IP. Always extract real IP from this header for rate limiting.

Response headers you must set

Content-Type — always set this explicitly. Never leave it to default.

Content-Type: application/json; charset=utf-8

X-Request-ID — echo the incoming request ID (or generate one if absent). The client and your logs can then correlate exactly which request produced this response.

Security headers — non-negotiable

These headers defend against entire classes of attacks. Install helmet in Express (or equivalent) and it sets most of them automatically. But understand what each one does.

Strict-Transport-Security (HSTS)

Strict-Transport-Security: max-age=31536000; includeSubDomains

Tells the browser: "Never connect to this domain over HTTP, always use HTTPS — for the next year." Prevents SSL stripping attacks where an attacker downgrades your HTTPS connection to HTTP. Once sent, the browser enforces this even if you later remove the header.

X-Content-Type-Options

X-Content-Type-Options: nosniff

Prevents browsers from "MIME sniffing" — guessing the content type when you don't set it. Without this, a browser might execute a file you served as text/plain as JavaScript if it looks like JS. Always set this.

X-Frame-Options

X-Frame-Options: DENY

Prevents your page from being loaded inside an <iframe>. Blocks clickjacking attacks — where an attacker embeds your site invisibly over their page and tricks users into clicking things.

Content-Security-Policy (CSP)

Content-Security-Policy: default-src 'self'; script-src 'self'

Tells the browser exactly which sources scripts, styles, images, etc. can be loaded from. The most powerful XSS defence available at the HTTP layer. Complex to configure correctly — start with default-src 'self' and add exceptions deliberately.

Referrer-Policy

Referrer-Policy: strict-origin-when-cross-origin

Controls how much referrer information is sent when a user follows a link from your site to another. Prevents leaking internal URLs or sensitive query params to third parties.

Permissions-Policy

Permissions-Policy: camera=(), microphone=(), geolocation=()

Explicitly disables browser features you don't use. If your API server doesn't need camera access, say so. Reduces attack surface.

Cache-Control for sensitive responses

Cache-Control: no-store

On any response containing sensitive data (user profile, financial data, tokens), set no-store. Prevents proxies and browsers from caching it. Without this, a shared computer might serve the previous user's data to the next user.

Part 3: REST API Design — The Contract With Your Clients

REST (Representational State Transfer) is not a protocol. It is a set of constraints for designing APIs on top of HTTP. An API that follows REST conventions is predictable — any developer who has used one REST API can figure out yours in minutes.

Resources are nouns, methods are verbs

Your URL identifies a resource. The HTTP method declares the action. Never put verbs in URLs.

# Wrong
GET  /getUser/42
POST /createUser
POST /deleteUser/42
POST /updateUserEmail

# Right
GET    /users/42
POST   /users
DELETE /users/42
PATCH  /users/42

The resource (/users/42) stays constant. What you do with it changes based on the method.

URL structure

Collections and items:

/users          → the collection of all users
/users/42       → a specific user
/users/42/posts → all posts belonging to user 42
/users/42/posts/7 → a specific post belonging to user 42

Rules:

• Lowercase only. /users not /Users
• Hyphens for spaces, not underscores. /blog-posts not /blog_posts
• No trailing slash. /users not /users/
• No file extensions. /users not /users.json
• Plural for collections. /users not /user

HTTP methods and their contracts

Idempotent means: calling it 10 times has the same result as calling it once. DELETE /users/42 called 5 times — first call deletes, rest return 404. The server state after all 5 calls is identical to after 1 call. This property is critical for retry logic on network failures.

Safe means: it has no side effects. It reads, never writes. GET is safe — calling it a thousand times changes nothing on the server.

Why this matters: If a client sends a POST and the network drops before they get a response, they don't know if the server received it. They cannot safely retry (it might create a duplicate). If it were PUT or DELETE (idempotent), they could safely retry because the result is the same regardless.

Response shape — be consistent across your entire API

Every response should follow the same structure. Pick one and commit to it everywhere.

// Success
{
  "data": { "id": 42, "name": "Trilochan" },
  "meta": { "requestId": "abc-123" }
}

// Collection
{
  "data": [ ... ],
  "meta": {
    "total": 100,
    "page": 1,
    "pageSize": 20,
    "requestId": "abc-123"
  }
}

// Error
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Email is required",
    "details": [ { "field": "email", "issue": "required" } ]
  },
  "meta": { "requestId": "abc-123" }
}

The rules:

• data always contains the resource(s)
• error is always an object, never a string
• error.code is a machine-readable constant (for the client to switch on)
• error.message is human-readable (for developers, maybe users)
• meta always contains the requestId
• Never mix data and error in the same response

Pagination

Never return unbounded collections. /users with 10 million users will kill your database.

Two approaches:

Offset pagination (simple, most common):

GET /users?page=2&limit=20

{
  "data": [ ... ],
  "meta": { "total": 1000, "page": 2, "pageSize": 20 }
}

Problem: OFFSET 100000 LIMIT 20 in SQL is slow — the database still scans 100,000 rows to skip them.

Cursor pagination (scalable, for large datasets):

GET /users?cursor=eyJpZCI6NDJ9&limit=20

{
  "data": [ ... ],
  "meta": { "nextCursor": "eyJpZCI6NjJ9", "hasMore": true }
}

The cursor encodes the position (e.g., base64 of { "id": 42 }). The query becomes WHERE id > 42 LIMIT 20 — a fast index scan regardless of position. Use this for any dataset that might grow large.

API versioning

APIs change. Clients don't always update immediately. Versioning lets you change the API without breaking existing clients.

URL versioning (most common, most explicit):

/v1/users
/v2/users

Header versioning (cleaner URLs but less visible):

Accept: application/vnd.yourapi.v2+json

Rules for versioning:

• A new version is only needed for breaking changes — removing fields, changing field types, changing behaviour
• Adding new fields, new endpoints — non-breaking, no new version needed
• Deprecate old versions with a Deprecation header and a sunset date
• Support the old version for at least 6 months after announcing deprecation

Part 4: The Request-Response Lifecycle — What Actually Happens

When a request hits your server, it passes through a pipeline. Understanding this pipeline is the foundation for understanding middleware, auth, and error handling.

Client Request
     ↓
[Nginx / Load Balancer]     ← terminates SSL, forwards to Node
     ↓
[Node.js Server]
     ↓
[Request ID Middleware]     ← attach unique ID to every request
     ↓
[Logging Middleware]        ← log request start (method, path, time)
     ↓
[Security Headers]          ← set helmet headers on every response
     ↓
[CORS Middleware]           ← handle cross-origin checks
     ↓
[Rate Limiter]              ← reject if too many requests from this IP/user
     ↓
[Body Parser]               ← parse JSON body into req.body
     ↓
[Input Validation]          ← reject if request shape is wrong (before any DB call)
     ↓
[Authentication]            ← verify token, attach user to req.user
     ↓
[Authorization]             ← check if this user can do this action
     ↓
[Route Handler]             ← your actual business logic
     ↓
[Error Handler]             ← catches any thrown error, returns structured response
     ↓
[Logging Middleware]        ← log response (status, duration)
     ↓
Client Response

This order is not arbitrary. Every position is deliberate:

• Request ID before everything — every log line from this point can reference it
• Rate limiting before auth — you don't want attackers hammering your auth layer; reject them before you hit the database
• Body parsing before validation — you need to parse before you can validate
• Validation before auth — reject obviously wrong requests before you do the expensive work of verifying a JWT and hitting the database
• Auth before your route handler — your route handler must be able to trust req.user exists
• Error handler last — it needs to catch errors from everything above

Part 5: Middleware — The Pipeline Pattern

Middleware is a function that sits in the request-response pipeline. It receives the request, does something, then either passes control to the next middleware or ends the cycle by sending a response.

In Express, middleware has this exact signature:

function middleware(req, res, next) {
  // do something
  next(); // pass to next middleware
  // OR
  res.json({ error: ... }); // end the cycle
}

The next() function is a contract. When you call next(), you are saying "I am done, pass this to whoever is registered after me." When you call res.json() or res.send(), you end the cycle — nothing after you runs (for this request).

Why middleware exists

Without middleware, every route handler would have to repeat the same logic:

// Without middleware — repeated in every handler
app.get('/users', (req, res) => {
  // verify JWT
  // check rate limit
  // parse body
  // log request
  // ... then actual logic
});

app.get('/posts', (req, res) => {
  // verify JWT (again)
  // check rate limit (again)
  // ... actual logic
});

Middleware moves cross-cutting concerns (auth, logging, rate limiting) out of route handlers and into reusable pipeline stages.

Applying middleware at different scopes

// Global — runs on every single request
app.use(requestId());
app.use(helmet());
app.use(cors(corsOptions));

// Route group — runs only on /api/* routes
const apiRouter = express.Router();
apiRouter.use(authenticate);
apiRouter.use(rateLimiter);

// Single route — runs only on this one route
app.post('/admin/nuke', requireRole('superadmin'), nukeHandler);

The error middleware — special signature

Error-handling middleware takes four parameters, not three. Express identifies it by the arity:

// This MUST be registered last — after all routes
app.use((err, req, res, next) => {
  // handle the error
  res.status(err.status || 500).json({
    error: {
      code: err.code || 'INTERNAL_ERROR',
      message: err.message
    }
  });
});

When any middleware or route handler calls next(err) — passing an argument to next — Express skips all regular middleware and jumps straight to the nearest error handler.

// In any middleware or route handler
try {
  const user = await db.users.findById(id);
  if (!user) throw new NotFoundError('User not found');
  next();
} catch (err) {
  next(err); // jumps straight to error middleware
}

Operational errors vs programmer errors

This distinction is critical for error handling:

Operational errors — expected failures that happen in a correct program. User not found, validation failed, rate limit exceeded, database timeout. These should be caught and returned as structured 4xx responses.

Programmer errors — bugs. Calling .map() on undefined, accessing a property on null, wrong argument type. These should crash the process (or be caught by your global error handler and logged as 500s with full stack traces for you to fix).

Never try to recover from a programmer error gracefully. You don't know what state the application is in. Log it, alert on it, and fix it.

Part 6: Input Validation — Trust Nothing

The single rule of input validation: never trust anything that comes from the client.

This includes:

• Request body (req.body)
• URL parameters (req.params)
• Query strings (req.query)
• Headers (req.headers)

All of it is attacker-controlled. Validate before it touches your business logic or database.

What validation means

Validation is checking that the input matches the shape and constraints you expect:

• Type — is this actually a string? A number? A boolean?
• Presence — is this field actually present and not null/undefined?
• Format — does this email look like an email? Is this UUID a valid UUID?
• Range — is this number between 1 and 100? Is this date in the future?
• Length — is this string under 255 characters?

Use a schema validation library. Do not hand-write validation logic. In TypeScript/Node.js the standard choices are Zod (best for TypeScript), Joi, or Yup.

import { z } from 'zod';

const CreateUserSchema = z.object({
  email: z.string().email(),
  password: z.string().min(8).max(128),
  name: z.string().min(1).max(100),
  role: z.enum(['user', 'admin']).default('user'),
});

// In middleware
function validateBody(schema) {
  return (req, res, next) => {
    const result = schema.safeParse(req.body);
    if (!result.success) {
      return res.status(400).json({
        error: {
          code: 'VALIDATION_ERROR',
          message: 'Invalid request body',
          details: result.error.errors.map(e => ({
            field: e.path.join('.'),
            issue: e.message
          }))
        }
      });
    }
    req.body = result.data; // use the parsed, typed data
    next();
  };
}

// Usage
app.post('/users', validateBody(CreateUserSchema), createUser);

What validation does not do

Validation checks shape. It does not sanitize for injection attacks. A valid email string can still contain SQL injection. Validation and injection prevention are different concerns — validation happens first, injection prevention happens at the database layer (parameterised queries always).

Validate at the boundary, not deep inside

Validate at the entry point of your application — the middleware layer — before any business logic. If your route handler has to check if (!email) return..., validation is in the wrong place.

Once data passes validation, your business logic should be able to assume the data is correct and typed.

Part 7: Authentication — Proving Identity

Authentication answers one question: who are you?

It is the process of verifying that the entity making a request is who they claim to be.

Sessions vs tokens — the fundamental choice

Session-based auth:

1. User logs in with credentials
2. Server creates a session in the database/cache: session_id → { userId: 42, role: 'admin' }
3. Server sends Set-Cookie: session_id=abc123; HttpOnly; Secure
4. Browser automatically sends cookie with every subsequent request
5. Server looks up session in store on every request

Token-based auth (JWT):

1. User logs in with credentials
2. Server creates a signed JWT containing: { userId: 42, role: 'admin', exp: ... }
3. Server sends token to client
4. Client stores token (memory or localStorage) and sends it manually: Authorization: Bearer <token>
5. Server verifies the token's signature on every request — no database lookup needed

The critical difference: sessions require a database lookup on every request. JWTs do not. The JWT's validity is proved by its cryptographic signature alone.

The tradeoff: JWTs cannot be invalidated before they expire. If you log out, the token is still valid until its expiry. Sessions can be deleted immediately.

JWT anatomy and internals

A JWT is three base64url-encoded JSON objects separated by dots:

header.payload.signature

Header:

{ "alg": "HS256", "typ": "JWT" }

Payload (claims):

{
  "sub": "42",          // subject — the user ID
  "role": "admin",      // custom claim
  "iat": 1716000000,    // issued at (unix timestamp)
  "exp": 1716604800     // expires at (unix timestamp)
}

Signature:

HMACSHA256(
  base64url(header) + "." + base64url(payload),
  secret
)

The server verifies the signature using its secret key. If even one character of the header or payload is changed, the signature won't match. The payload is not encrypted — just base64-encoded. Anyone can decode it. Never put sensitive data (passwords, PII) in a JWT payload.

Algorithm choice — HS256 vs RS256

HS256 (HMAC-SHA256): Uses a single shared secret. Both signing and verification use the same key. Simple. Use this when only your server signs and verifies tokens.

RS256 (RSA-SHA256): Uses a private key to sign and a public key to verify. Different keys for signing vs verification. Use this when you have multiple services that need to verify tokens — they get the public key only, never the private key.

Implementing JWT auth correctly

import jwt from 'jsonwebtoken';

const ACCESS_TOKEN_SECRET = process.env.ACCESS_TOKEN_SECRET;
const REFRESH_TOKEN_SECRET = process.env.REFRESH_TOKEN_SECRET;

// On login
function generateTokens(user) {
  const accessToken = jwt.sign(
    { sub: user.id, role: user.role },
    ACCESS_TOKEN_SECRET,
    { expiresIn: '15m' } // short-lived
  );

  const refreshToken = jwt.sign(
    { sub: user.id },
    REFRESH_TOKEN_SECRET,
    { expiresIn: '7d' } // long-lived
  );

  return { accessToken, refreshToken };
}

// Store refresh token in database (so you can invalidate it)
await db.refreshTokens.create({ userId: user.id, token: refreshToken });

The dual-token pattern is mandatory for production:

• Access token — short-lived (15 minutes). Sent in every API request header. If stolen, expires quickly.
• Refresh token — long-lived (7 days). Stored in database. Used only to get a new access token. Can be revoked in the database.

When the access token expires, the client sends the refresh token to /auth/refresh. If the refresh token is valid and exists in the database, issue a new access token. This gives you the best of both worlds: stateless verification on most requests, revocability when needed.

Auth middleware

async function authenticate(req, res, next) {
  const authHeader = req.headers.authorization;

  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({
      error: { code: 'MISSING_TOKEN', message: 'Authentication required' }
    });
  }

  const token = authHeader.slice(7);

  try {
    const payload = jwt.verify(token, ACCESS_TOKEN_SECRET);
    req.user = { id: payload.sub, role: payload.role };
    next();
  } catch (err) {
    if (err.name === 'TokenExpiredError') {
      return res.status(401).json({
        error: { code: 'TOKEN_EXPIRED', message: 'Token has expired' }
      });
    }
    return res.status(401).json({
      error: { code: 'INVALID_TOKEN', message: 'Invalid token' }
    });
  }
}

Critical implementation rules:

• Max access token lifetime: 15-30 minutes. Never 7 days for an access token.
• Max refresh token lifetime: 7 days. Rotate on every use (issue new, invalidate old).
• Refresh tokens must be stored in the database — so they can be revoked on logout or compromise.
• Use HttpOnly; Secure; SameSite=Strict cookies for storing refresh tokens in browser clients. Never localStorage for security-critical tokens.
• Always return 401, never 403, when the token is missing or invalid.

Part 8: Authorization & RBAC — Controlling What You Can Do

Authorization answers a different question from authentication: what are you allowed to do?

Authentication: "You are Trilochan." (identity) Authorization: "Trilochan can read posts but cannot delete users." (permissions)

They are completely separate concerns. Authentication comes first. Authorization comes second, after you know who you are dealing with.

RBAC — Role-Based Access Control

RBAC is the most common authorization model. Instead of assigning permissions directly to users, you assign permissions to roles, and assign roles to users.

User → Role → Permissions

For example:

• Role viewer → can read posts, read own profile
• Role editor → can read and write posts, read own profile
• Role admin → can do everything

This scales. If you have 10,000 users and you want to add a new permission to all editors, you add it to the editor role — done. Without RBAC, you'd update 10,000 rows.

Designing your permission model

Define permissions as resource:action strings:

users:read
users:create
users:update
users:delete
posts:read
posts:create
posts:update:own     // can update their own posts only
posts:update:any     // can update any post
posts:delete:own
posts:delete:any

// In your database
const ROLES = {
  viewer: ['posts:read', 'users:read'],
  editor: ['posts:read', 'posts:create', 'posts:update:own', 'posts:delete:own'],
  admin:  ['*'], // all permissions
};

Implementing authorization middleware

// Check if user has required permission
function requirePermission(permission) {
  return (req, res, next) => {
    const userRole = req.user.role;
    const permissions = ROLES[userRole] || [];

    const hasPermission =
      permissions.includes('*') ||
      permissions.includes(permission);

    if (!hasPermission) {
      return res.status(403).json({
        error: {
          code: 'FORBIDDEN',
          message: 'You do not have permission to perform this action'
        }
      });
    }

    next();
  };
}

// Usage
app.delete('/posts/:id',
  authenticate,                          // who are you?
  requirePermission('posts:delete:any'), // can you do this?
  deletePost
);

Ownership checks — the subtlety most people miss

RBAC at the role level is not enough. You also need to check ownership at the data level.

// An 'editor' can delete:own posts — but only their own
app.delete('/posts/:id',
  authenticate,
  async (req, res, next) => {
    const post = await db.posts.findById(req.params.id);

    if (!post) {
      return res.status(404).json({ error: { code: 'NOT_FOUND' } });
    }

    const isOwner = post.authorId === req.user.id;
    const isAdmin = req.user.role === 'admin';

    if (!isOwner && !isAdmin) {
      return res.status(403).json({
        error: { code: 'FORBIDDEN', message: 'You can only delete your own posts' }
      });
    }

    next();
  },
  deletePost
);

Critical security rule: Fetch the resource first, then check ownership. Never rely on the client sending the owner ID — they can forge it. The ground truth is in your database.

IDOR — Insecure Direct Object Reference

The most common authorization bug. A user requests /invoices/1042 — but invoice 1042 belongs to another user. Your code finds the invoice (it exists), and returns it because you didn't check ownership.

// WRONG — IDOR vulnerability
app.get('/invoices/:id', authenticate, async (req, res) => {
  const invoice = await db.invoices.findById(req.params.id);
  res.json({ data: invoice }); // returns anyone's invoice
});

// CORRECT
app.get('/invoices/:id', authenticate, async (req, res) => {
  const invoice = await db.invoices.findOne({
    id: req.params.id,
    userId: req.user.id  // always scope to the authenticated user
  });

  if (!invoice) {
    return res.status(404).json({ error: { code: 'NOT_FOUND' } });
  }

  res.json({ data: invoice });
});

Always scope database queries to the authenticated user when the resource is user-owned. Never fetch first, then check — fetch with the constraint.

Part 9: Error Handling — Failing Gracefully and Informatively

Most backends handle errors wrong. They either leak internal details (stack traces, SQL errors, internal paths) or return useless generic messages that give the client no way to help the user.

The error hierarchy

Create a hierarchy of custom error classes:

// Base error class
class AppError extends Error {
  constructor(message, code, status) {
    super(message);
    this.name = 'AppError';
    this.code = code;      // machine-readable: 'NOT_FOUND', 'VALIDATION_ERROR'
    this.status = status;  // HTTP status code: 404, 400, 500
    this.isOperational = true; // vs programmer errors
  }
}

class NotFoundError extends AppError {
  constructor(message = 'Resource not found') {
    super(message, 'NOT_FOUND', 404);
  }
}

class ValidationError extends AppError {
  constructor(message, details = []) {
    super(message, 'VALIDATION_ERROR', 400);
    this.details = details;
  }
}

class UnauthorizedError extends AppError {
  constructor(message = 'Authentication required') {
    super(message, 'UNAUTHORIZED', 401);
  }
}

class ForbiddenError extends AppError {
  constructor(message = 'Access denied') {
    super(message, 'FORBIDDEN', 403);
  }
}

class ConflictError extends AppError {
  constructor(message) {
    super(message, 'CONFLICT', 409);
  }
}

The global error handler

// Must be registered LAST — after all routes
app.use((err, req, res, next) => {
  // Log all errors with context
  logger.error({
    message: err.message,
    code: err.code,
    stack: err.stack,
    requestId: req.id,
    method: req.method,
    path: req.path,
    userId: req.user?.id,
  });

  // Operational error — expected, safe to return details
  if (err.isOperational) {
    return res.status(err.status).json({
      error: {
        code: err.code,
        message: err.message,
        ...(err.details && { details: err.details }),
      },
      meta: { requestId: req.id }
    });
  }

  // Programmer error — unexpected bug. Never leak details.
  // Alert your on-call. Return generic message.
  return res.status(500).json({
    error: {
      code: 'INTERNAL_ERROR',
      message: 'An unexpected error occurred',
    },
    meta: { requestId: req.id }
  });
});

The rules:

• Never send stack traces to the client in production
• Never send database error messages to the client (column "emial" does not exist tells an attacker your schema)
• Always include requestId in error responses so users can report it and you can find it in logs
• Always log the full error (stack, context) server-side, even when returning a generic message to the client

Async error handling

Express does not catch async errors automatically in older versions. Wrap async handlers or use a wrapper:

// The wrapper pattern — cleaner
const asyncHandler = (fn) => (req, res, next) =>
  Promise.resolve(fn(req, res, next)).catch(next);

app.get('/users/:id', asyncHandler(async (req, res) => {
  const user = await db.users.findById(req.params.id);
  if (!user) throw new NotFoundError('User not found');
  res.json({ data: user });
}));

The .catch(next) automatically passes any thrown error to your error middleware.

Part 10: Security — Defending the Surface

CORS — Cross-Origin Resource Sharing

Why it exists: Browsers enforce a security model called the Same-Origin Policy. A script running on evil.com cannot make requests to yourapi.com and read the responses. CORS is the mechanism that lets you selectively lift this restriction for trusted origins.

CORS only applies to browser-based requests. curl, Postman, mobile apps, server-to-server requests — none of them are subject to CORS. CORS is a browser protection, not a server protection.

How it works:

For cross-origin requests with non-simple methods (POST, PUT, DELETE, PATCH) or custom headers, the browser sends a preflight request first:

OPTIONS /api/users HTTP/1.1
Origin: https://yourfrontend.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-Type, Authorization

Your server must respond:

Access-Control-Allow-Origin: https://yourfrontend.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization
Access-Control-Max-Age: 86400

Only then does the browser send the actual request.

Correct CORS configuration:

import cors from 'cors';

const ALLOWED_ORIGINS = [
  'https://yourapp.com',
  'https://app.yourapp.com',
  process.env.NODE_ENV === 'development' ? 'http://localhost:3000' : null,
].filter(Boolean);

app.use(cors({
  origin: (origin, callback) => {
    // Allow requests with no origin (mobile apps, curl, Postman)
    if (!origin) return callback(null, true);

    if (ALLOWED_ORIGINS.includes(origin)) {
      return callback(null, true);
    }

    callback(new Error(`CORS blocked: ${origin} not allowed`));
  },
  credentials: true,           // Allow cookies to be sent cross-origin
  methods: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS'],
  allowedHeaders: ['Content-Type', 'Authorization', 'X-Request-ID'],
  maxAge: 86400,               // Cache preflight for 24 hours
}));

Never do this:

app.use(cors({ origin: '*', credentials: true }));
// This is invalid — wildcard origin cannot be used with credentials
// Some browsers will reject this silently

Never use wildcard * for authenticated APIs. * means any origin in the world can make requests to your API in a browser context. Acceptable for truly public data (CDN fonts, public analytics). Never for any API with auth.

Rate Limiting

Rate limiting prevents abuse. Without it, a single attacker can:

• Enumerate all user IDs (/users/1, /users/2, ...)
• Brute-force passwords on your login endpoint
• DDoS your server into the ground
• Drain your third-party API budget (OpenAI, SMS, email)

The token bucket algorithm (conceptually):

Each user/IP has a "bucket" with a maximum capacity of N tokens. Each request consumes one token. Tokens refill at a constant rate. When the bucket is empty, requests are rejected with 429. This allows bursting (a user can make several quick requests) while enforcing a long-term average rate.

Implementation:

import rateLimit from 'express-rate-limit';
import RedisStore from 'rate-limit-redis';
import { redisClient } from './redis.js';

// Global limit — applies to everything
const globalLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 500,                   // 500 requests per window
  standardHeaders: true,      // Return X-RateLimit-* headers
  legacyHeaders: false,
  store: new RedisStore({ client: redisClient }), // Distributed — works across multiple servers
  keyGenerator: (req) => req.headers['x-forwarded-for'] || req.ip,
  handler: (req, res) => {
    res.status(429).json({
      error: { code: 'RATE_LIMIT_EXCEEDED', message: 'Too many requests' }
    });
  }
});

// Strict limit for auth endpoints — brute force prevention
const authLimiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 10, // only 10 login attempts per 15 minutes
  store: new RedisStore({ client: redisClient }),
  keyGenerator: (req) => req.body.email || req.ip, // per email address, not just IP
});

app.use(globalLimiter);
app.post('/auth/login', authLimiter, loginHandler);
app.post('/auth/forgot-password', authLimiter, forgotPassword);

Use Redis as the store — not in-memory. In-memory rate limiting only works on a single Node process. If you run 4 Node processes (cluster), each has its own counter — the limit is effectively 4x higher. Redis is shared across all processes.

Always return Retry-After in the 429 response so clients know when to try again.

SQL Injection Prevention

SQL injection happens when untrusted input is concatenated into a SQL query:

// NEVER DO THIS
const user = await db.query(
  `SELECT * FROM users WHERE email = '${req.body.email}'`
);
// Input: ' OR '1'='1
// Query becomes: SELECT * FROM users WHERE email = '' OR '1'='1'
// Returns every user in the database

Prevention is simple: always use parameterised queries or an ORM.

// Parameterised query — safe
const user = await db.query(
  'SELECT * FROM users WHERE email = $1',
  [req.body.email]  // database driver handles escaping
);

// Prisma ORM — safe by default
const user = await prisma.user.findUnique({
  where: { email: req.body.email }
});

The database driver treats the parameter as data, never as SQL code. No amount of injection in the parameter can change the query structure.

Password Handling

Passwords must never be stored in plain text or with reversible encryption. Use a slow, adaptive hashing algorithm.

import bcrypt from 'bcrypt';

const SALT_ROUNDS = 12; // Higher = slower = harder to brute force

// On registration
const hashedPassword = await bcrypt.hash(password, SALT_ROUNDS);
await db.users.create({ email, password: hashedPassword });

// On login
const isValid = await bcrypt.compare(inputPassword, user.password);
if (!isValid) throw new UnauthorizedError('Invalid credentials');

Why bcrypt:

• Includes salt automatically (prevents rainbow table attacks)
• Is deliberately slow (each verification takes ~100-300ms, making brute force prohibitively expensive)
• SALT_ROUNDS = 12 is a good production default — increase over time as hardware gets faster

Never:

• Store plain text passwords
• Use MD5 or SHA1 for passwords (they're fast — bad for passwords)
• Return whether the email or password was wrong separately ("email not found" vs "wrong password") — this tells attackers which emails are registered. Always return "Invalid credentials".

Environment Variables and Secrets

// Never hardcode secrets
const secret = 'my-jwt-secret'; // NEVER

// Always use environment variables
const secret = process.env.JWT_SECRET;

// Validate on startup — fail fast if secrets are missing
const requiredEnvVars = [
  'JWT_SECRET',
  'DATABASE_URL',
  'REDIS_URL',
  'REFRESH_TOKEN_SECRET'
];

for (const key of requiredEnvVars) {
  if (!process.env[key]) {
    throw new Error(`Missing required environment variable: ${key}`);
  }
}

Rules:

• .env files are never committed to git (add to .gitignore before first commit — always)
• Rotate secrets every 90 days
• Use different secrets for every environment (dev/staging/prod)
• Use a secrets manager (AWS Secrets Manager, Doppler, Vault) in production — not just .env files on a server

Part 11: Logging — Seeing What Happens in Production

You cannot debug what you cannot see. Logging is the visibility layer of your backend.

Structured logging

Never use console.log. Use a proper logger that outputs JSON.

import pino from 'pino';

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  // In production: output raw JSON, pipe to log aggregator
  // In development: use pino-pretty for human-readable output
});

export default logger;

Structured logs are JSON objects, not strings. The difference matters enormously at scale:

// Wrong — a string you can't query
console.log(`User 42 failed login from 192.168.1.1`);

// Right — a JSON object you can filter, aggregate, alert on
logger.warn({
  event: 'login_failed',
  userId: 42,
  ip: '192.168.1.1',
  requestId: req.id,
  reason: 'invalid_password',
});

In your log aggregator (Datadog, Loki, CloudWatch), you can then query: "show me all login_failed events in the last hour grouped by IP" — impossible with plain string logs.

Request ID — the thread through everything

Every request gets a unique ID. It is attached to every log line generated during that request's lifecycle.

import { randomUUID } from 'crypto';

function requestId(req, res, next) {
  // Use the incoming ID if present (from client or upstream service)
  // Generate one if absent
  req.id = req.headers['x-request-id'] || randomUUID();

  // Echo it back in the response — client can include it in bug reports
  res.setHeader('X-Request-ID', req.id);

  next();
}

// Request logging middleware
function requestLogger(req, res, next) {
  const start = Date.now();

  logger.info({
    event: 'request_received',
    requestId: req.id,
    method: req.method,
    path: req.path,
    ip: req.headers['x-forwarded-for'] || req.ip,
    userAgent: req.headers['user-agent'],
  });

  res.on('finish', () => {
    logger.info({
      event: 'request_completed',
      requestId: req.id,
      method: req.method,
      path: req.path,
      statusCode: res.statusCode,
      durationMs: Date.now() - start,
      userId: req.user?.id,
    });
  });

  next();
}

Now when a user reports "I got an error, here's my request ID: abc-123", you can find every log line from that exact request in seconds.

What to log and what not to log

Always log:

• Every request and response (method, path, status, duration)
• Auth events (login, logout, failed login, token refresh)
• Errors (with full stack trace and context)
• Business events (user created, payment processed, etc.)
• External service calls (database queries > 100ms, third-party API calls)

Never log:

• Passwords or password hashes
• Full JWT tokens
• Credit card numbers, CVVs
• National IDs, SSNs
• Full request bodies for auth endpoints
• API keys or secrets

Log levels — use them correctly

ERROR — something broke, needs immediate attention
WARN  — something unexpected but handled (wrong password, rate limit hit)
INFO  — normal events you want a record of (request, user created)
DEBUG — verbose detail for debugging (only enabled locally)

Set LOG_LEVEL=info in production. LOG_LEVEL=debug floods your logs and costs money.

Part 12: Health Checks — Knowing Your System is Alive

Load balancers and orchestration systems (Kubernetes, PM2, Docker) need to know if your server is healthy. Health check endpoints let them automatically remove unhealthy instances and route traffic away from them.

Two types of health checks

Liveness — is the process running at all? A liveness check should never fail unless the process is completely broken. It just confirms the server can respond.

app.get('/health/live', (req, res) => {
  res.json({ status: 'ok', uptime: process.uptime() });
});

Readiness — is the service ready to handle traffic? A readiness check verifies all dependencies are reachable. If your database is down, the service is not ready.

app.get('/health/ready', async (req, res) => {
  const checks = {};

  // Check database
  try {
    await db.$queryRaw`SELECT 1`;
    checks.database = 'ok';
  } catch {
    checks.database = 'error';
  }

  // Check Redis
  try {
    await redis.ping();
    checks.redis = 'ok';
  } catch {
    checks.redis = 'error';
  }

  const allHealthy = Object.values(checks).every(v => v === 'ok');

  res.status(allHealthy ? 200 : 503).json({
    status: allHealthy ? 'ok' : 'degraded',
    checks,
    timestamp: new Date().toISOString(),
  });
});

The distinction matters: A liveness probe failure restarts the container. A readiness probe failure removes the instance from the load balancer rotation without restarting it. Restarting won't fix a database that's down — you just want to stop sending traffic to that instance.

Shallow vs deep checks

Shallow — just checks if the server process is up. Use for liveness.

Deep — checks all dependencies. Use for readiness. But be careful: if your deep check itself becomes slow or fails in an unexpected way, it can cause cascading problems. Add a timeout to every dependency check.

async function checkWithTimeout(checkFn, timeoutMs = 3000) {
  return Promise.race([
    checkFn(),
    new Promise((_, reject) =>
      setTimeout(() => reject(new Error('Timeout')), timeoutMs)
    )
  ]);
}

Health check security

Health endpoints should not require authentication (load balancers need to call them). But they also shouldn't expose sensitive internal information.

// Fine to expose
{ status: 'ok', checks: { database: 'ok', redis: 'ok' } }

// Never expose
{ databaseUrl: 'postgres://...', internalIp: '10.0.0.42', version: '1.2.3' }

If you want a detailed health check with version info for internal use, add a separate /health/detailed endpoint that requires an internal API key.

Part 13: API Contracts — Making Your API Predictable

An API contract is the complete, unambiguous specification of what your API accepts and what it returns. It is the agreement between you and every client of your API.

OpenAPI / Swagger specification

The industry standard for documenting REST APIs. Write your contract in code, generate documentation from it.

# openapi.yaml
openapi: 3.0.0
info:
  title: Your API
  version: 1.0.0

paths:
  /users/{id}:
    get:
      summary: Get a user by ID
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
            format: uuid
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
        '404':
          $ref: '#/components/responses/NotFound'

components:
  schemas:
    User:
      type: object
      required: [id, email, name, createdAt]
      properties:
        id:
          type: string
          format: uuid
        email:
          type: string
          format: email
        name:
          type: string
        createdAt:
          type: string
          format: date-time

Benefits: clients can generate SDK code from it. Generates interactive documentation. Can be used to automatically validate requests and responses.

Response consistency — the last rule

Enforce these rules in every single route, without exception:

1. Always include requestId in every response (success and error)
2. Always use the same top-level shape (data for success, error for failure, meta for metadata)
3. Always include createdAt and updatedAt in resource representations — clients need to know when data changed
4. Always return arrays inside data, never a bare array at the top level (makes it impossible to add metadata later without a breaking change)
5. Never change a field's type between versions without a major version bump
6. Never remove a field without a major version bump — you can add, never remove

Putting It All Together

Here is what a production-ready Express application structure looks like, applying every principle in this document:

src/
├── middleware/
│   ├── requestId.js       ← first in pipeline
│   ├── requestLogger.js   ← second
│   ├── security.js        ← helmet + CORS
│   ├── rateLimiter.js     ← before body parsing
│   ├── authenticate.js    ← verify JWT
│   └── errorHandler.js    ← last
├── validators/
│   ├── user.validators.js ← Zod schemas per resource
│   └── auth.validators.js
├── routes/
│   ├── auth.routes.js
│   ├── user.routes.js
│   └── health.routes.js
├── services/
│   └── user.service.js    ← business logic, no HTTP knowledge
├── errors/
│   └── AppError.js        ← error hierarchy
├── lib/
│   ├── logger.js          ← pino logger
│   ├── redis.js
│   └── db.js
└── app.js

// app.js — the pipeline in order
app.use(requestId());
app.use(requestLogger());
app.use(helmet());
app.use(cors(corsOptions));
app.use(globalRateLimiter);
app.use(express.json({ limit: '10kb' })); // body size limit prevents payload attacks
app.use('/health', healthRouter);          // health checks before auth
app.use('/auth', authLimiter, authRouter);
app.use('/api', authenticate, apiRouter);  // all /api routes require auth
app.use(errorHandler);                     // last

Quick Reference

The invariants — never violate these

• Never store plain text passwords
• Never put secrets in code or git
• Never trust input from the client — validate everything
• Never return stack traces to the client in production
• Never use wildcard CORS with credentials
• Never skip ownership checks — always scope queries to the authenticated user
• Never return 200 with an error in the body
• Never build rate limiting on in-memory state in a multi-process environment
• Never log sensitive data (passwords, tokens, PII)
• Never access req.body before running the body parser middleware

Status codes to commit to memory

Middleware order

requestId → logger → security headers → CORS → rate limiter
→ body parser → validation → authenticate → authorize
→ route handler → error handler

The checklist for every new endpoint

Before shipping any new endpoint, confirm:

• Input validated with schema (params, body, query)
• Authentication required (if not public)
• Authorization checked (role and ownership)
• Database query scoped to authenticated user where applicable
• Correct HTTP method (GET reads, POST creates, etc.)
• Correct status code returned
• Error response follows standard shape
• Sensitive operations have stricter rate limits
• No secrets or internal paths in response body
• Logs written with requestId and userId context

Last principle: these are not rules to memorise. They are consequences of understanding how HTTP, browsers, databases, and attackers work. If you understand the why behind each section, you can derive the what in any situation you haven't seen before.