Phase 5 · Chapter 4

System Design: WhatsApp

Real-time messaging, delivery guarantees, and the hardest problem — effectively-once delivery

What this chapter does to you: WhatsApp looks like a simple chat app. It is one of the hardest system design problems because it combines real-time delivery, offline users, message ordering, presence detection, and end-to-end encryption — all at 2 billion users. The bottleneck is not storage or bandwidth. It is connection state — how do you know if a user is online, and how do you guarantee a message reaches them exactly once? After this chapter, you will understand WebSockets at the protocol level, why message queues exist per-user, what "delivered" vs "read" actually means in a distributed system, and how end-to-end encryption works without the server seeing your messages.

Step 1: Clarify Requirements

Functional requirements:

One-to-one messaging between users
Group messaging (up to 1,024 members)
Message delivery to offline users (store and forward)
Message status: sent → delivered → read (the three ticks)
Media: photos, videos, voice messages, documents
Presence: online / offline / last seen
End-to-end encryption
Push notifications when app is backgrounded

Non-functional requirements:

2 billion registered users, 500 million DAU
Low latency: messages should arrive in < 100ms when both users are online
High availability: messaging must work even during partial infrastructure outages
Message ordering: messages from the same sender must arrive in order
Effectively-once delivery: a message must not be lost and must not be shown twice (often called "exactly-once" — but see Part 5 for why that name is misleading)
Durability: a sent message must not be lost, even if recipient is offline for days

The core tensions:

"Exactly-once" + "retry on failure" = these two requirements fight each other
"Low latency" + "durability" = fast means RAM, durable means disk
"End-to-end encrypted" + "server stores messages" = the server cannot read what it stores

These tensions drive every architectural decision in this chapter.

Step 2: Estimate Scale

Messages:

500M DAU, each sends ~40 messages/day
20 billion messages/day = ~230,000 messages/second
Peak: ~500,000 messages/second

Media:

~30% of messages contain media
Average media: 500KB
6 billion media messages/day × 500KB = 3 petabytes of media per day
Media is NOT stored forever — WhatsApp deletes media after 30 days if not downloaded

Connections:

500M concurrent online users at peak
Each user holds one persistent WebSocket connection
500 million open TCP connections = the infrastructure problem of this system

Presence updates:

Each user sends a "heartbeat" every 30 seconds to signal online status
500M users × 2 heartbeats/minute = ~16 million presence updates/minute
~270,000 presence ops/second — a separate high-frequency write problem

What this tells you:

500M concurrent WebSocket connections cannot be served by one machine. A single server handles ~65,000 TCP connections (port range limit). You need ~8,000 connection servers minimum.
Message volume (230k/sec) is high but manageable with queuing.
Media storage at 3PB/day → object storage only, never a DB.
Presence at 270k/sec → Redis, never Postgres.

Part 3: WebSockets — The Protocol Under the Hood

HTTP is request-response: client asks, server answers, connection closes. But messaging requires the server to push to the client without the client asking first.

Three options exist. Know all three.

Option 1: HTTP Polling

javascript

// Client polls every 2 seconds
setInterval(async () => {
  const messages = await fetch('/api/messages/new')
  displayMessages(messages)
}, 2000)

Simple. Works. Universally terrible for messaging.

500M users × 30 requests/minute = 15 billion HTTP requests/minute
Each request: TCP + TLS handshake overhead (from Chapter 2)
Most responses: empty ("no new messages")
Latency: up to 2 seconds to see a new message

Used only as a last resort fallback when nothing else works.

Option 2: Long Polling

Client sends a request. Server holds it open until a message arrives or timeout.

javascript

// Client
async function longPoll() {
  const response = await fetch('/api/messages/wait?timeout=30')
  if (response.ok) displayMessages(await response.json())
  longPoll()   // immediately reconnect
}

Better than polling. Message arrives within milliseconds. But: each "waiting" request is a held HTTP connection — same TCP overhead. At 500M users, 500M held HTTP connections. Not fundamentally different from WebSockets in connection count, but without the efficiency of the WebSocket protocol.

Option 3: WebSocket — The Right Tool

WebSocket starts as HTTP, upgrades to a persistent bidirectional channel.

code

Client → Server:
GET /chat HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==

Server → Client:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

After the 101 Switching Protocols, the TCP connection stays open. Both client and server can send frames at any time — no HTTP headers, no repeated handshakes, just lightweight framed binary data.

A WebSocket frame overhead: 2-14 bytes. An HTTP request overhead: ~500 bytes minimum. For 230,000 messages/second, this difference is enormous.

The physical reality:

One TCP connection per user, stays open as long as the app is active
Server can push messages to client without client asking
Heartbeat frames (tiny pings) keep the connection alive through NATs and firewalls
On mobile: connection drops when app backgrounds → push notifications take over

Part 4: The Connection Server — Managing 500M WebSockets

One machine cannot hold 500M WebSocket connections. Linux's default limit is 65,535 ports. With tuning, a single server can hold ~100,000-500,000 concurrent connections.

Connection servers (also called gateway servers):

Each connection server holds N WebSocket connections and knows:

Which user is connected to it (user ID → connection mapping)
Nothing else — it's a dumb pipe

The routing problem:

User A (on Server 1) sends a message to User B (on Server 2).

Server 1 receives the message from A's WebSocket. How does it know to deliver it to Server 2? It can't — it doesn't know where B is connected.

Solution: a routing layer

The Presence Service stores which connection server each user is on:

code

HSET user:connections
  userA "server-1"
  userB "server-2"
  userC "server-1"
  ...

# Or per user:
SET user:B:connection_server "server-2" EX 60
# TTL 60 seconds — refreshed by heartbeat
# Expired = user is offline

This Redis key is the critical piece. Every message delivery looks up this key. At 230,000 messages/second = 230,000 Redis reads/second just for routing. Redis can handle this. One Redis cluster for routing state.

Part 5: Message Delivery — The Hard Part

The Three States of a Message

WhatsApp shows three states for every message:

code

✓   Sent       — your server received it
✓✓  Delivered  — recipient's device received it
✓✓  Read       — recipient opened and read it (blue ticks)

These are not cosmetic. Each tick represents a different guarantee:

Single tick (Sent): Your message left your phone and was accepted by the WhatsApp server. It's persisted. Even if recipient is offline for a week, it will be delivered.

Double tick (Delivered): The message was pushed to the recipient's device and their app acknowledged receipt. The recipient's phone has the message. Even if they lose internet next second, they have it.

Blue ticks (Read): The recipient opened the conversation containing this message. Their app sent a read receipt back to the server, which forwarded it to you.

Implementing this requires the server to track message state transitions.

Message State Machine

Storage:

sql

CREATE TABLE messages (
  id            BIGINT PRIMARY KEY,   -- Snowflake ID, generated by the server
  client_msg_id UUID NOT NULL,        -- generated by the sender's device (idempotency key)
  sender_id     BIGINT NOT NULL,
  recipient_id  BIGINT NOT NULL,      -- or group_id for groups
  content       BYTEA NOT NULL,       -- encrypted blob, server can't read
  media_url     TEXT,                 -- if media message
  sent_at       TIMESTAMP NOT NULL,
  delivered_at  TIMESTAMP,            -- null until delivered
  read_at       TIMESTAMP             -- null until read
);

CREATE INDEX idx_messages_recipient ON messages(recipient_id, sent_at)
  WHERE delivered_at IS NULL;         -- partial index: only undelivered messages

-- One row per (sender, client message id): a retried send can never create a second row
CREATE UNIQUE INDEX idx_messages_dedup ON messages(sender_id, client_msg_id);

The partial index — only undelivered messages — is the key optimization. When User B comes online, you query "all undelivered messages for User B." This index makes that query hit only the small fraction of undelivered messages, not the entire message history.

The Exactly-Once Delivery Problem

User A sends a message. Network drops after server receives it but before ACK returns to A. A doesn't know if the server got it. A retries.

Naive result: server gets the message twice. User B sees duplicates.

First, the honest framing. True exactly-once delivery over an unreliable network is impossible: the sender can never tell "the server didn't get it" apart from "the server got it but the ACK was lost." What real systems build is:

at-least-once delivery (keep retrying until ACKed) + an idempotent receiver (a duplicate is detected and ignored) = effectively-once processing.

Say it that way in an interview. Claiming "exactly-once delivery" without that qualifier is a red flag.

Solution: idempotency keys, checked by the durable store

javascript

// Client generates a unique ID for each message before sending.
// A random UUID, not a Snowflake: Snowflake needs a unique worker ID,
// which millions of phones don't have.
const clientMsgId = crypto.randomUUID()

socket.send(JSON.stringify({
  type: 'message',
  id: clientMsgId,        // idempotency key, reused on every retry
  to: 'userB',
  content: encryptedContent
}))

Server receives message:

javascript

async function handleIncomingMessage(senderId, msg) {
  // 1. Durable write FIRST. The unique index on (sender_id, client_msg_id)
  //    makes a retried message a no-op instead of a second row.
  const result = await db.query(
    `INSERT INTO messages (id, client_msg_id, sender_id, recipient_id, content, sent_at)
     VALUES ($1, $2, $3, $4, $5, now())
     ON CONFLICT (sender_id, client_msg_id) DO NOTHING
     RETURNING id`,
    [generateSnowflake(), msg.id, senderId, msg.to, msg.content]
  )
  const isNew = result.rowCount === 1

  // 2. ACK only after the message is safely stored.
  //    New or duplicate, the answer to the sender is the same: "I have it."
  socket.send({ type: 'ack', id: msg.id })

  // 3. Hand off delivery only for the first copy.
  if (isNew) await deliverMessage(result.rows[0].id)
}

Why the order matters — the bug to avoid. A common version marks the ID as seen first (e.g. SET msg:dedup:{id} 1 NX in Redis) and stores the message after. If the server crashes between those two steps, every retry hits "already seen → ACK" and the message is never stored. You turned at-least-once into at-most-once and lost a message silently. Rule: record "done" in the same atomic step as the work itself, or after it — never before.

A Redis SET NX check is still useful as a fast path in front of the database (skip the insert for obvious duplicates), but only if it is written after the durable insert succeeds, and the database constraint stays the source of truth.

Client only removes the message from its send queue after receiving ACK. If ACK never comes: retry with the same id. Server deduplicates. Result: effectively-once — no loss, no duplicates — despite network failures. (Delivery to the recipient's device works the same way: the server retries until the device ACKs, and the device ignores message IDs it has already shown.)

This is the same idempotency principle from the Queues chapter (Phase 4, Chapter 4), applied to the messaging layer. The pattern is universal.

Delivery to Offline Users — The Mailbox Pattern

User B is offline. A message arrives for them.

The server cannot push to a closed WebSocket. So it stores the message. When B reconnects, the server drains the mailbox.

javascript

// When user connects
async function onUserConnect(userId, socket) {
  // Register connection
  await redis.set(`user:${userId}:server`, SERVER_ID, 'EX', 60)

  // Drain mailbox — fetch all undelivered messages
  const pending = await db.query(`
    SELECT * FROM messages
    WHERE recipient_id = $1
    AND delivered_at IS NULL
    ORDER BY sent_at ASC
  `, [userId])

  // Send all pending messages
  for (const msg of pending) {
    socket.send(JSON.stringify(msg))
  }

  // Mark as delivered
  await db.query(`
    UPDATE messages
    SET delivered_at = NOW()
    WHERE recipient_id = $1
    AND delivered_at IS NULL
  `, [userId])
}

The ordering guarantee: Messages are sent in sent_at ASC order — oldest first. The recipient sees messages in the order they were sent, even if they came from different senders and were queued for days.

Part 6: Group Messaging — Fan-out Revisited

Group messages are the Twitter fan-out problem applied to messaging.

User A sends a message to a group of 500 members. The server must deliver it to all 500 members.

javascript

async function deliverGroupMessage(message, groupId) {
  // Fetch all group members
  const members = await db.query(
    'SELECT user_id FROM group_members WHERE group_id = $1',
    [groupId]
  )

  // For each member: check online/offline, deliver or store
  await Promise.all(members.map(async (member) => {
    const server = await redis.get(`user:${member.userId}:server`)

    if (server) {
      // Online — push via connection server
      await deliverToServer(server, member.userId, message)
    } else {
      // Offline — store for later
      await storeForDelivery(member.userId, message)
    }
  }))
}

At 500 members, this is fine — 500 parallel operations.

The large group problem:

WhatsApp caps groups at 1,024 members. Why?

At 1,024 members, one message triggers 1,024 delivery operations. At 230,000 messages/second (many from groups), fan-out can reach 230,000 × 1,024 = 235 million delivery operations/second.

WhatsApp's cap is an engineering constraint, not an arbitrary product decision.

Write amplification for groups:

Unlike Twitter (one post → fan-out asynchronously), messaging has a latency requirement: recipient must see the message in < 100ms. Async fan-out via queue adds latency. For groups, WhatsApp does synchronous fan-out for online members, async for offline.

javascript

async function deliverGroupMessage(message, groupId) {
  const members = await getGroupMembers(groupId)

  const online = []
  const offline = []

  for (const member of members) {
    const server = await redis.get(`user:${member}:server`)
    if (server) online.push({ member, server })
    else offline.push(member)
  }

  // Online members: deliver synchronously (within the request)
  await Promise.all(online.map(({ member, server }) =>
    deliverToServer(server, member, message)
  ))

  // Offline members: queue for later
  await offlineQueue.add('deliverOffline', { message, recipients: offline })
}

Part 7: Presence — Who Is Online?

Presence is a high-frequency write problem.

Every active user sends a heartbeat every 30 seconds:

code

Client → Server: { type: 'heartbeat', userId: 'abc' }
Server: SET user:abc:last_seen NOW EX 60

TTL of 60 seconds: if no heartbeat in 60 seconds, key expires → user is offline. No explicit "I'm going offline" message needed — just stop heartbeating.

Reading presence:

User A opens a chat with User B. Is B online?

javascript

async function isOnline(userId) {
  const lastSeen = await redis.get(`user:${userId}:last_seen`)
  return lastSeen !== null   // key exists = online (not yet expired)
}

O(1) Redis read. Sub-millisecond.

The privacy problem:

WhatsApp has "last seen" privacy settings. Users can hide their last seen from non-contacts or from everyone.

javascript

async function getPresence(requesterId, targetUserId) {
  const privacy = await getPrivacySetting(targetUserId, 'last_seen')

  if (privacy === 'nobody') return null
  if (privacy === 'contacts') {
    const isContact = await areContacts(requesterId, targetUserId)
    if (!isContact) return null
  }

  return redis.get(`user:${targetUserId}:last_seen`)
}

This is an authorization check (Chapter 3, Phase 4) layered on top of a presence lookup.

Presence at group scale:

When you open a group with 500 members, you don't load all 500 presence statuses. WhatsApp shows "500 members" and only loads presence for the first visible members on scroll. Lazy loading presence is the correct approach — not because presence is slow, but because 500 Redis reads on every group open × millions of group opens/second = unnecessary load.

Part 8: End-to-End Encryption — What the Server Doesn't Know

End-to-end encryption (E2EE) means: only sender and recipient can read messages. The server stores and forwards encrypted blobs it cannot decrypt.

WhatsApp uses the Signal Protocol — the gold standard for E2EE messaging.

The Key Problem

Encryption requires a key. How do A and B agree on a shared secret key without the server knowing it — even though all communication goes through the server?

You already know the answer from Chapter 2 (TLS): Diffie-Hellman key exchange. The Signal Protocol uses a variant called X3DH (Extended Triple Diffie-Hellman).

How It Works (Simplified)

Step 1: Key generation (done on device, never shared with server)

Each device generates:

Identity key pair (IK): long-term. Proves "this is really user A."
Signed prekey pair (SPK): medium-term, rotated monthly.
One-time prekeys (OPK): batch of single-use keys.

Step 2: Upload public keys to server

code

User A uploads to WhatsApp server:
  - Public identity key
  - Public signed prekey + signature
  - 100 public one-time prekeys (a batch)

The server stores these. It cannot derive private keys from public keys.

Step 3: A wants to message B for the first time

A fetches B's public keys from the server:

code

A gets from server:
  - B's public identity key
  - B's public signed prekey
  - One of B's one-time prekeys (server marks it used, deletes from pool)

A uses X3DH with B's public keys + A's private keys → derives a shared secret. B, when they receive the first message, uses X3DH with A's public keys + B's private keys → derives the same shared secret.

Neither A nor B sent the shared secret to the other. Neither A nor B sent it to the server. The server never sees it.

This is the Diffie-Hellman magic from TLS chapter — applied to asynchronous messaging where B might be offline when A first messages them.

Step 4: Message encryption

javascript

// On A's device (simplified):
const sharedSecret = x3dh(A.privateKeys, B.publicKeys)
const messageKey = deriveMessageKey(sharedSecret, messageCounter)
const encryptedContent = aes256gcm_encrypt(plaintext, messageKey)

// Send to server:
{ to: 'userB', content: encryptedContent, counter: messageCounter }
// Server stores this blob — it cannot decrypt it

AES-256-GCM: same algorithm from TLS (Chapter 2). Symmetric, hardware-accelerated, fast.

Forward secrecy:

The Signal Protocol derives a new key for every message using a ratchet mechanism. Even if an attacker steals your current key, they cannot decrypt past messages (those used different keys that are now deleted) or future messages (those will use derived keys from the current state, which changes with each message).

This is called forward secrecy — compromise of current key doesn't compromise past or future.

What the Server Actually Stores

code

{
  id: "12345",
  sender: "userA",
  recipient: "userB",
  encrypted_content: <binary blob>,  ← server cannot decrypt
  sent_at: timestamp,
  delivered_at: null
}

WhatsApp knows: who messaged whom, when, how often. It cannot read what. This metadata is still significant — it reveals communication patterns even without message content. This is the argument for metadata-blind protocols (Tor, Signal's sealed sender) which are beyond this chapter's scope.

Part 9: Push Notifications — When the App is Backgrounded

User B's phone is in their pocket. App is backgrounded or killed. No WebSocket connection. A sends a message.

The server stores the message (mailbox pattern). But how does B know to open the app and fetch it?

Push notification infrastructure:

code

WhatsApp Server → Apple APNs (for iOS) → User B's iPhone
WhatsApp Server → Google FCM (for Android) → User B's Android

APNs (Apple Push Notification Service) and FCM (Firebase Cloud Messaging) are Apple and Google's infrastructure for waking up backgrounded apps.

WhatsApp never stores your push notification token on its own — it's handled via APNs/FCM who know how to reach specific devices.

The notification payload:

json

{
  "aps": {
    "alert": "New message",
    "badge": 3,
    "sound": "default"
  },
  "messageId": "12345"
}

The notification does NOT contain message content (that would bypass E2EE). It says "you have a new message." The app opens, connects WebSocket, drains mailbox, decrypts messages on device.

The race condition:

User B gets a push notification. Opens app. Connects WebSocket. Receives messages. Meanwhile, the server was also retrying the push notification. B might receive two notifications for the same message.

Fix: push notification has a messageId. App checks local message store. If messageId already received: ignore the duplicate notification. Idempotency again.

Part 10: The Complete Architecture

The data flow for a message — both online:

The data flow for a message — B is offline:

Part 11: The Vocabulary Map

Word	What it actually is
`WebSocket`	Persistent bidirectional TCP connection. HTTP upgrade to full-duplex channel.
`Long polling`	Client waits with open HTTP request until server has data. Better than polling, worse than WS.
`Connection server`	Gateway server holding N WebSocket connections. Stateless except for open sockets.
`Presence`	Online/offline status. Redis key with TTL, refreshed by heartbeat.
`Heartbeat`	Periodic ping from client. Keeps WebSocket alive, refreshes presence TTL.
`Mailbox`	Per-user store of undelivered messages. Drained on reconnect.
`Exactly-once delivery`	Impossible over a lossy network. What systems build is effectively-once: at-least-once delivery + idempotent receiver.
`Idempotency key`	Client-generated unique ID (UUID) per message. Server deduplicates on it via a unique constraint, before ACKing.
`Message ACK`	Recipient's device confirms it received the message. Triggers double tick.
`Read receipt`	Recipient opened the conversation. Triggers blue ticks.
`Forward secrecy`	Compromise of current key doesn't expose past/future messages.
`E2EE`	End-to-end encryption. Server stores encrypted blobs it cannot decrypt.
`Signal Protocol`	Gold-standard E2EE protocol used by WhatsApp and Signal.
`X3DH`	Extended Triple Diffie-Hellman. Asynchronous key agreement — works when B is offline.
`Ratchet`	Key derivation mechanism. New key per message. Old keys deleted.
`APNs / FCM`	Apple / Google push notification infrastructure. Wakes backgrounded apps.
`Fan-out`	Delivering one message to N group members. Write amplification problem.
`Routing table`	Redis: user → connection server. Consulted on every message delivery.

Part 12: Practice Problems

Problem 1 (Scale Reasoning): WhatsApp limits groups to 1,024 members. A viral group has 1,024 members. All 1,024 are online simultaneously. One member sends a message every 5 seconds.

Calculate:

Messages/second for this one group
Connection server reads/second (routing lookups)
WebSocket write operations/second
If you have 10,000 such groups active simultaneously, what is the total Redis ops/second for routing alone?

At what group size would a single Redis instance (200k ops/sec) become the bottleneck for routing?

Problem 2 (Exactly-Once Design): Suppose that instead of the database unique constraint, your only dedup check is a Redis key: SET msg:dedup:{messageId} 1 NX EX 86400

A message is sent at 11:59pm. The 24-hour TTL expires at 11:59pm the next day. The same user's app retries the same message at 12:01am two days later (extreme edge case — app was offline for 25 hours, then reconnected).

What happens? Is this acceptable? How would you fix it? What does this tell you about choosing TTLs for idempotency keys?

Problem 3 (Architecture — Offline Delivery): User B is offline for 7 days (phone dead, on vacation). They have 3,000 undelivered messages waiting in Postgres.

When they reconnect:

How do you deliver 3,000 messages without overwhelming their device?
In what order do you send them?
What happens if their device ACKs message 1,500 and then disconnects again?
How do you ensure they get messages 1,501-3,000 without re-sending 1-1,500?

Design the exact protocol — what the server sends, what the client ACKs, how state is tracked.

Problem 4 (E2EE + Backup): WhatsApp lets you back up your messages to Google Drive or iCloud. But messages are E2EE — the server can't decrypt them.

How does backup work without breaking E2EE? Who holds the key to decrypt the backup? What is the security tradeoff when you enable cloud backup? (This is a real architectural decision WhatsApp made — reason from first principles before looking it up.)

Problem 5 (Connect Everything): Design the "message search" feature for WhatsApp. Users can search their own chat history for a keyword.

Constraints:

Messages are E2EE — server cannot index them
A user might have 5 years of message history
Search must return results in < 500ms
The search index cannot be stored on the server (E2EE constraint)

Where does the search index live? How is it built? How do you search 5 years of messages on a mobile device in < 500ms? What happens when the user gets a new phone?

This problem has no textbook answer — reason from the constraints.

Where We Are in Phase 5

code

Phase 5 — System Design HLD
  ✅ Chapter 1: URL Shortener   (read-heavy, caching, ID generation)
  ✅ Chapter 2: Twitter Feed    (fan-out, write amplification, social graph)
  ✅ Chapter 3: YouTube         (media pipeline, CDN, adaptive streaming)
  ✅ Chapter 4: WhatsApp        (real-time, E2EE, effectively-once delivery)
  ⬜ Chapter 5: Uber            (geospatial, real-time matching, surge pricing)
  ⬜ Chapter 6: Google Docs     (real-time collaboration, OT vs CRDT)
  ⬜ Chapter 7: Notifications   (fan-out at scale, multi-channel delivery)

Three chapters left in Phase 5.

Chapter 5 is Uber — the geospatial problem. The bottleneck is neither storage, bandwidth, connection state, nor fan-out. It is location data at high write frequency — millions of drivers updating their GPS coordinates every 4 seconds, millions of riders querying "find me the nearest available driver."

The hard problem: how do you efficiently query "who is within 500 meters of me" on a data set that changes completely every 4 seconds? This requires a data structure you haven't seen yet: geohashing.

End of Phase 5, Chapter 4. Estimated reading time: 80-95 minutes. Practice problems: 60-90 minutes. Do not proceed to Chapter 5 until you can answer Problem 3 — the 3,000-message offline delivery protocol. It requires you to design a stateful resumable delivery mechanism from scratch. If you can design it cleanly — what the server sends, what the client ACKs, how partial delivery is tracked — you understand distributed state machines. That is the hardest concept in this entire curriculum.