Q65 — Design a news feed / timeline (Twitter or Instagram style)

What the system must do

When a user opens the app, they see a feed of posts from accounts they follow, sorted by recency (or a ranking algorithm). When a user posts, their followers see it in their feeds. The core operations:

• Write: user posts content → it should appear in followers' feeds
• Read: user opens app → sees ranked list of recent posts from followees

Scale targets: 300M daily active users, average 200 followers per user, 100M posts per day, reads vastly outnumber writes (the feed is read dozens of times per session).

The fundamental design tension: fan-out on write vs fan-out on read

This is the central architectural decision of a news feed. There are two opposing approaches.

Fan-out on write (push model): when a user posts, immediately write the post ID into every follower's pre-built feed (a list stored per user). When a follower opens the app, their feed is already computed — just read it.

Fan-out on read (pull model): when a user opens the app, query all the accounts they follow, fetch recent posts from each, merge and rank. The feed is computed at read time.

Each has a critical failure mode:

Fan-out on write fails for celebrities. Kylie Jenner has 300M followers. One post triggers 300M feed-write operations — a write storm that overwhelms the system. These are called "hotspot celebrities" or "heavy hitters."

Fan-out on read fails for high-follower-count users' readers. If you follow 5,000 accounts and each has recent posts, fan-out on read means merging 5,000 sorted lists at read time — expensive and slow.

The production solution is a hybrid: fan-out on write for regular users, fan-out on read for celebrities.

Data model

-- Posts
CREATE TABLE posts (
  id         BIGINT PRIMARY KEY,  -- Snowflake ID (sortable by time)
  user_id    BIGINT NOT NULL,
  content    TEXT,
  created_at TIMESTAMPTZ NOT NULL
);

-- Follow graph
CREATE TABLE follows (
  follower_id BIGINT,
  followee_id BIGINT,
  PRIMARY KEY (follower_id, followee_id)
);

-- Pre-built feed (fan-out on write target)
-- Stored in Redis sorted set per user
-- ZADD user:{user_id}:feed <timestamp_score> <post_id>

The feed for a user is stored in Redis as a sorted set: key = feed:{user_id}, members = post IDs, score = timestamp. ZREVRANGE returns the most recent N posts.

Fan-out on write — the write path

When user 42 (with 1,000 followers) posts:

1. Post is written to the posts table, gets a Snowflake ID
2. A Kafka event post_created is published with {user_id: 42, post_id: ...}
3. The feed fan-out worker consumes the event
4. Worker fetches follower list from the follows table for user 42: 1,000 follower IDs
5. For each follower: ZADD feed:{follower_id} <post_timestamp> <post_id> in Redis
6. Trim the sorted set to the last 1,000 entries (you don't need infinite feed history in the fast store)

1,000 Redis writes per post. At 100M posts/day, this is 100 billion Redis writes per day — manageable with a Redis cluster because these are small, atomic operations spread across the cluster by key.

Handling celebrities — the fan-out on read path

Define a "celebrity" threshold: any user with more than 1M followers is a celebrity. When a celebrity posts, skip the fan-out worker. Don't pre-build the feed update.

At read time, when building user A's feed:

1. Start with the pre-built Redis feed (populated by fan-out on write from all non-celebrity followees)
2. Identify which followees of user A are celebrities
3. For each celebrity followee, fetch their recent posts directly from the posts table (or a per-celebrity Redis cache)
4. Merge the pre-built feed with celebrity posts, re-rank, and return the top N

The merge is at most merging a handful of celebrity streams with the pre-built feed — manageable at read time.

This hybrid eliminates both failure modes: no write storm for celebrities, no expensive read-time merge for regular users.

Feed ranking

A purely chronological feed is simple but suboptimal. Posts from 3 days ago (that you missed) might be more relevant than low-quality posts from 5 minutes ago. Modern feeds use a ranking model:

• Recency (base score)
• Engagement signals: likes, comments, shares on the post so far
• Relationship strength: you interact with this account frequently
• Content type preference: videos vs photos vs text

The ranking model runs offline (batch or near-real-time) to precompute scores. The feed shows posts sorted by score, not raw time. Heavy ranking computation happens asynchronously — the served feed has pre-ranked post IDs, not raw timestamps.

Read path

1. User opens app → GET /feed?user_id=42&cursor=...
2. Server fetches ZREVRANGE feed:42 0 49 from Redis → 50 post IDs (sorted by score or timestamp)
3. For each post ID, fetch post content (from Redis post cache or DB)
4. Merge with celebrity posts (step above)
5. Return to client

Pagination uses cursor-based pagination (Q22): the cursor is the score/timestamp of the last seen post. "Give me posts with score < cursor."

Post content is cached in Redis: key = post:{post_id}, value = JSON of the post. Cache hit on virtually all reads — posts are written once and read many times.

The mental model

Core tension: fan-out on write vs fan-out on read
  Write: push post to all followers' feeds at write time → fast read, write storm for celebrities
  Read: pull and merge at read time → always fresh, slow for users following many accounts
  Hybrid: write for regular users (<1M followers), read-time merge for celebrities

Data:
  posts table (Postgres) — source of truth
  feed:{user_id} sorted set (Redis) — pre-built feed, score = timestamp/rank
  post:{post_id} cache (Redis) — post content for fast retrieval
  follows table — follow graph (may be graph DB at extreme scale)

Write path:
  Post → DB → Kafka(post_created) → fan-out worker → ZADD to each follower's feed
  Skip celebrities → their posts fetched at read time

Read path:
  ZREVRANGE feed:{user_id} → post IDs
  Fetch content from post cache
  Merge with celebrity posts for followees exceeding threshold
  Return sorted/ranked list

Ranking: offline scoring model → pre-scored post IDs in feed sorted set

Pagination: cursor-based (score/timestamp of last seen post)

Scale:
  100B fan-out writes/day → Redis cluster sharded by user_id
  Read path entirely from Redis (sub-5ms) → no DB touch on hot path
  Celebrity threshold = architectural parameter (tune based on write load)

A news feed is one of the most elegantly difficult systems to design: the fan-out problem only reveals itself when you think through celebrities, the hybrid model is the non-obvious answer, and almost every concept in this question bank — caching, consistent hashing, Snowflake IDs, cursor pagination, Kafka, and consumer groups — applies in one place.