Phase 5 · Chapter 2

System Design: Twitter / Social Feed

The fan-out problem — when one write must become ten million reads

What this chapter does to you: The feed system is the canonical write-amplification problem in system design. Every social product — Twitter, Instagram, LinkedIn, YouTube subscriptions — faces the same fundamental question: when someone with 10 million followers posts something, how do you update 10 million feeds without destroying your infrastructure? After this chapter, you will understand fan-out on write vs fan-out on read, why Twitter uses a hybrid, and how to reason about any social system's feed architecture from scratch.

Step 1: Clarify Requirements

Functional requirements:

• Users can post tweets (up to 280 characters, optional media)
• Users can follow / unfollow other users
• Users see a home feed: tweets from people they follow, in reverse chronological order
• Users can like, retweet, reply
• Search (out of scope for this chapter — separate system)
• Notifications (out of scope — separate system)

Non-functional requirements:

• 500 million daily active users (DAU)
• High availability: feed must load even during partial outages
• Low latency: feed must load in < 200ms
• Eventual consistency: a user might see a new tweet 1-2 seconds after it's posted. Acceptable.
• Write latency: posting a tweet should feel instant to the poster. Followers seeing it can have slight delay.

The core tension you just identified: "Posting should feel instant" + "500 million users" + "200ms feed load" = there is a fundamental conflict between write speed and read speed at this scale. Every design decision in this chapter is about managing that conflict.

Step 2: Estimate Scale

Tweets:

• 500M DAU, each user posts ~0.5 tweets/day average
• 250 million tweets/day = ~2,900 tweets/second
• Peak: ~7,000 tweets/second

Feed reads:

• Each user checks feed ~10 times/day
• 500M × 10 = 5 billion feed loads/day
• ~58,000 feed loads/second
• Peak: ~150,000 feed loads/second

Follow relationships:

• Average user follows ~200 accounts
• 500M users × 200 follows = 100 billion follow relationships
• This is a massive graph. Storage: ~100B × 8 bytes (two 64-bit user IDs) = ~800GB

Media:

• ~30% of tweets have media (photos, videos)
• 2,900 tweets/sec × 0.3 = ~870 media uploads/second
• Media stored on separate object storage (S3/R2), not in DB

What this tells you:

• Reads (150k/sec) far exceed writes (7k/sec) — 20:1 ratio
• Follow graph is enormous — can't be stored naively
• Feed construction is the hard problem — not storage, not tweets themselves

Step 3: The Core Problem — Feed Construction

When user A opens their feed, what do you show them?

The conceptually simple query:

SELECT tweets.*
FROM tweets
JOIN follows ON tweets.user_id = follows.followed_id
WHERE follows.follower_id = 'userA'
ORDER BY tweets.created_at DESC
LIMIT 20

This query joins two massive tables. At scale:

• follows table: 100 billion rows
• tweets table: hundreds of billions of rows
• This query, even with perfect indexes, could take seconds
• At 150,000 requests/second — catastrophic

This query cannot run on demand at this scale. Full stop.

So how do you show users their feed?

Two fundamentally different architectures exist.

Part 4: Fan-out on Write — Pre-computed Feeds

The Idea

When user A posts a tweet, immediately push it to all of A's followers' feeds. Each user has a pre-built feed in Redis.

The read path is trivially fast:

async function getFeed(userId) {
  // Feed is pre-built in Redis as a Sorted Set (score = tweet timestamp)
  const tweetIds = await redis.zrevrange(`feed:${userId}`, 0, 19)
  const tweets = await Promise.all(tweetIds.map(id => getTweet(id)))
  return tweets
}

One Redis call. 20 tweet lookups (also cached). Feed loads in < 10ms.

The write path is expensive:

async function postTweet(userId, content) {
  // 1. Save tweet
  const tweetId = await db.insert('tweets', { userId, content })

  // 2. Get all followers — could be millions
  const followers = await db.query(
    'SELECT follower_id FROM follows WHERE followed_id = $1',
    [userId]
  )

  // 3. Push to each follower's feed
  // This is a QUEUE job — we don't do this synchronously
  await fanoutQueue.add('fanout', { tweetId, followers: followers.map(f => f.id) })

  // 4. Return immediately — poster doesn't wait for fanout
  return tweetId
}

The fanout worker:

worker = new Worker('fanout', async (job) => {
  const { tweetId, followers } = job.data
  const timestamp = Date.now()

  // Pipeline: batch all Redis writes in one round-trip
  const pipeline = redis.pipeline()
  for (const followerId of followers) {
    pipeline.zadd(`feed:${followerId}`, timestamp, tweetId)
    pipeline.zremrangebyrank(`feed:${followerId}`, 0, -1001)  // keep only latest 1000
  }
  await pipeline.exec()
})

The Thundering Herd: Celebrity Accounts

Fan-out on write breaks catastrophically for celebrities.

Cristiano Ronaldo has 160 million Twitter followers. He posts a tweet. Your system must do 160 million Redis writes.

160 million writes × ~0.5KB per write = 80GB of Redis writes from one tweet. At Redis throughput of ~200,000 ops/second = 800 seconds to complete. 13 minutes of backlog from one tweet.

During those 13 minutes, his followers start checking their feed. They don't see his new tweet yet. Your fanout queue is drowning. Meanwhile, 10 more celebrities post. The queue never catches up.

This is the fundamental problem with pure fan-out on write: It scales with follower count. For celebrities, it doesn't scale at all.

Part 5: Fan-out on Read — Compute on Demand

The Idea

Don't pre-compute feeds. When a user opens their feed, compute it on the spot.

async function getFeed(userId) {
  // 1. Get list of who this user follows (cached in Redis)
  const following = await redis.smembers(`following:${userId}`)

  // 2. For each followed user, get their recent tweets
  const tweetIdSets = await Promise.all(
    following.map(id => redis.zrevrange(`user_tweets:${id}`, 0, 99))
  )

  // 3. Merge all tweet ID lists, sort by timestamp, take top 20
  const allTweetIds = tweetIdSets.flat()
  allTweetIds.sort((a, b) => b.score - a.score)
  const topTweetIds = allTweetIds.slice(0, 20)

  // 4. Fetch tweet content
  return Promise.all(topTweetIds.map(id => getTweet(id)))
}

Advantages:

• Write path is trivial: just save the tweet and add to author's own tweet list
• No fan-out cost — Kylie Jenner posts, one Redis write
• Feed is always perfectly up-to-date

Disadvantages:

• User A follows 1,000 accounts
• Feed load = 1,000 Redis calls (one per followed account) + merge + sort
• Even in parallel, 1,000 Redis lookups per feed load is expensive
• At 150,000 feed loads/second: 150 million Redis ops/second just for feed construction
• That's 750 Redis instances running at capacity

Pure fan-out on read doesn't scale either — it pushes the cost onto reads instead of writes.

Part 6: The Hybrid — What Twitter Actually Does

Twitter (and most production social systems) use a hybrid:

Fan-out on write for regular users. Fan-out on read for celebrities.

The threshold: if a user has < N followers (say, 1 million), fan-out on write. If a user has > N followers, skip fan-out on write entirely.

const CELEBRITY_THRESHOLD = 1_000_000

async function postTweet(userId, content) {
  const tweetId = await saveTweet(userId, content)

  // Always: add to author's own timeline (for their profile page)
  await redis.zadd(`user_tweets:${userId}`, Date.now(), tweetId)

  const followerCount = await getFollowerCount(userId)

  if (followerCount < CELEBRITY_THRESHOLD) {
    // Regular user: fan-out on write
    await fanoutQueue.add('fanout', { tweetId, userId })
  }
  // Celebrity: skip fan-out. Followers will pull on read.

  return tweetId
}

The read path for the hybrid:

async function getFeed(userId) {
  // 1. Get pre-built feed (contains tweets from non-celebrities)
  const prebuiltFeed = await redis.zrevrangebyscore(
    `feed:${userId}`, '+inf', '-inf',
    'WITHSCORES', 'LIMIT', 0, 100
  )

  // 2. Get celebrities this user follows
  const followedCelebrities = await getCelebritiesFollowed(userId)

  // 3. For each celebrity, get their recent tweets (fan-out on read)
  const celebrityTweets = await Promise.all(
    followedCelebrities.map(id =>
      redis.zrevrange(`user_tweets:${id}`, 0, 19)
    )
  )

  // 4. Merge prebuilt feed + celebrity tweets, sort, return top 20
  const merged = mergeAndSort([prebuiltFeed, ...celebrityTweets])
  return merged.slice(0, 20)
}

The user follows 500 regular people and 5 celebrities.

• Prebuilt feed: 1 Redis call (contains 500 regular people's tweets)
• Celebrity tweets: 5 Redis calls (one per celebrity)
• Total: 6 Redis calls to construct feed. Fast.

Why this works: Most users are regular users. Fan-out on write handles them efficiently. Celebrities are rare. 5 extra Redis calls per feed load is manageable. The catastrophic 160M-write scenario is avoided.

Step 7: The Data Model

Tweets Table

CREATE TABLE tweets (
  id           BIGSERIAL PRIMARY KEY,
  user_id      BIGINT NOT NULL,
  content      VARCHAR(280) NOT NULL,
  media_urls   TEXT[],              -- array of S3/R2 URLs
  reply_to_id  BIGINT,              -- null if original tweet
  retweet_id   BIGINT,              -- null if original tweet
  created_at   TIMESTAMP DEFAULT NOW(),
  like_count   INT DEFAULT 0,       -- denormalized counter
  retweet_count INT DEFAULT 0,      -- denormalized counter
  reply_count  INT DEFAULT 0
);

CREATE INDEX idx_tweets_user_id ON tweets(user_id, created_at DESC);
CREATE INDEX idx_tweets_created_at ON tweets(created_at DESC);

Follows Table — The Social Graph

CREATE TABLE follows (
  follower_id  BIGINT NOT NULL,
  followed_id  BIGINT NOT NULL,
  created_at   TIMESTAMP DEFAULT NOW(),
  PRIMARY KEY (follower_id, followed_id)
);

CREATE INDEX idx_follows_follower ON follows(follower_id);   -- "who do I follow?"
CREATE INDEX idx_follows_followed ON follows(followed_id);   -- "who follows me?"

At 100 billion rows, this table is 800GB+. It cannot fit in RAM. Queries against it must use indexes. Even with indexes, at 500M users querying simultaneously, this becomes a bottleneck.

The fix: cache the social graph

// Cache who a user follows — small enough per user, frequently read
async function getFollowing(userId) {
  const cached = await redis.smembers(`following:${userId}`)
  if (cached.length > 0) return cached

  const following = await db.query(
    'SELECT followed_id FROM follows WHERE follower_id = $1 LIMIT 5000',
    [userId]
  )
  await redis.sadd(`following:${userId}`, ...following.map(f => f.followed_id))
  await redis.expire(`following:${userId}`, 3600)

  return following.map(f => f.followed_id)
}

Cap at 5,000 for the cache. If a user follows > 5,000 accounts, fall back to DB for the long tail. Edge case, tolerable.

User Feed in Redis

Key: feed:{userId}
Type: Sorted Set
Score: tweet timestamp (Unix milliseconds)
Member: tweet ID

ZADD feed:userB 1705312800000 "tweet:12345"
ZADD feed:userB 1705312900000 "tweet:12346"
ZREVRANGE feed:userB 0 19        → latest 20 tweet IDs

Keep maximum 800 tweets per user feed. Beyond that, if the user scrolls that far back, compute on demand from DB. Keeps Redis memory bounded.

800 tweets × 500M users × ~20 bytes per ZSet entry = 8TB

8TB of Redis is expensive. In practice, only cache feeds for active users. Users inactive for > 7 days: evict feed from Redis. Recompute on next login.

Tweet Cache in Redis

// Each tweet cached individually as a Hash
HSET tweet:12345
  userId      "456"
  content     "Hello world"
  likeCount   "1423"
  createdAt   "1705312800000"

// Cache for 24 hours — tweets don't change (content immutable)
// Like counts update asynchronously in background

Fetching 20 tweets for a feed = 20 parallel Redis HGET calls. With pipelining: one round-trip, 20 responses. Sub-millisecond.

Step 8: Likes — The Counter Problem

User likes a tweet. 1 billion likes/day on Twitter. ~11,600 like operations/second. Peak: ~30,000/second.

Naive approach:

INSERT INTO likes (user_id, tweet_id) VALUES ($1, $2);
UPDATE tweets SET like_count = like_count + 1 WHERE id = $1;

At 30,000 likes/second, UPDATE tweets SET like_count + 1 on hot tweets creates massive write contention. Every like on a viral tweet locks that row.

The fix: write-behind counter (Chapter 1, Phase 4)

async function likeTweet(userId, tweetId) {
  // 1. Check if already liked (idempotency)
  const alreadyLiked = await redis.sismember(`likes:${tweetId}`, userId)
  if (alreadyLiked) return

  // 2. Record like in Redis Set (deduplication)
  await redis.sadd(`likes:${tweetId}`, userId)

  // 3. Increment counter in Redis (atomic, instant)
  await redis.incr(`like_count:${tweetId}`)

  // 4. Queue DB write — async
  await likesQueue.add('recordLike', { userId, tweetId })
}

Background worker:

// Every 30 seconds: flush Redis counters to Postgres
async function flushLikeCounts() {
  const tweetIds = await redis.keys('like_count:*')
  for (const key of tweetIds) {
    const count = await redis.get(key)
    const tweetId = key.split(':')[1]
    await db.query(
      'UPDATE tweets SET like_count = $1 WHERE id = $2',
      [count, tweetId]
    )
  }
}

Like count shown to users comes from Redis — instant, accurate enough. Postgres count is slightly stale (< 30 seconds) — acceptable. If Redis fails, counters restore from Postgres on restart plus re-accumulate from queue.

The consistency tradeoff: A like might show up in Redis (real-time) before it's confirmed in Postgres. If Redis fails in that window, the like might appear to disappear. For like counts, this is acceptable. For something financial, it would not be.

Step 9: The Timeline Problem — Ordering with Snowflake IDs

A feed must be in reverse chronological order. Using created_at timestamps has a problem: two tweets can have the same millisecond. At 7,000 tweets/second, timestamp collisions are common.

Twitter's solution: Snowflake IDs.

A Snowflake ID is a 64-bit integer containing:

┌─────────────────────────────────────────────────────────────┐
│  1 bit     │  41 bits           │  10 bits    │  12 bits    │
│  unused    │  timestamp (ms)    │  machine ID │  sequence   │
│            │  since epoch       │             │  number     │
└─────────────────────────────────────────────────────────────┘

• 41-bit timestamp: milliseconds since a custom epoch (Jan 1, 2010 for Twitter). 41 bits = ~69 years of unique milliseconds.
• 10-bit machine ID: identifies the server that generated this ID. Up to 1,024 machines can generate IDs simultaneously.
• 12-bit sequence: within the same millisecond, same machine, up to 4,096 unique IDs.

Max throughput: 1,024 machines × 4,096 IDs/ms = ~4 billion IDs/second.

Why Snowflake solves the ordering problem: IDs are monotonically increasing within a machine because the timestamp is the most significant bits. Sorting by ID = sorting by time. No separate ORDER BY created_at needed. The ID itself encodes time.

function generateSnowflake(machineId) {
  const now = BigInt(Date.now() - EPOCH)  // ms since custom epoch
  const sequence = BigInt(nextSequence())  // 0-4095
  return (now << 22n) | (BigInt(machineId) << 12n) | sequence
}

// Extract timestamp from a Snowflake ID:
function getTimestamp(snowflakeId) {
  return Number(BigInt(snowflakeId) >> 22n) + EPOCH
}

Sorting tweet IDs numerically = sorting chronologically. Elegant.

Step 10: Search — Why It's a Separate System

When you search "ChatGPT" on Twitter, you don't run:

SELECT * FROM tweets WHERE content LIKE '%ChatGPT%' ORDER BY created_at DESC

At hundreds of billions of tweets, this query would never finish.

Full-text search on tweets uses Elasticsearch — a separate system built for exactly this purpose. Tweets are indexed into Elasticsearch as they're created. Elasticsearch maintains an inverted index: "ChatGPT" → [tweet IDs containing this word].

This is out of scope for this chapter but important to name: search is architecturally separate from feed. Different data store, different scaling model.

Step 11: The Complete Architecture

The flow for posting a tweet:

The flow for loading a feed:

Tradeoffs Summary

Practice Problems

Problem 1 (Scale — Fan-out Math): A user has 50,000 followers. They post 5 tweets per day.

• How many Redis writes does fan-out on write produce per day for this user?
• At what follower count does fan-out on write become the system's bottleneck? (Assume your fanout workers can do 500,000 Redis writes/second combined)
• If Twitter has 10,000 "celebrity" accounts (>1M followers) each posting 3 times/day, how many reads does fan-out on read add to feed construction? (Assume 10% of your 500M users follow at least one celebrity)

Problem 2 (Data Model): You want to support "lists" — a user creates a named list of accounts (e.g., "Tech CEOs") and can view a feed for just that list.

How does this change your:

• Data model (what tables/keys do you add?)
• Fan-out logic (do list tweets get fanned out like regular tweets?)
• Feed construction (how do you build a list feed?)
• Redis memory usage

Problem 3 (Failure Analysis): Your Redis cluster goes down. You have 500M users whose feed is stored in Redis.

Walk through exactly what happens when a user tries to load their feed. Design a graceful degradation strategy — users should see something, not a blank page. What is the minimum viable fallback that still gives users a useful feed?

Problem 4 (The Unfollow Problem): User A unfollows User B. User A's feed (pre-built in Redis) still contains User B's past tweets.

When should you remove User B's tweets from User A's feed? Should you do it immediately, lazily, or not at all? What are the tradeoffs for each approach? What does Twitter actually do? (Reason from first principles first.)

Problem 5 (Connect Everything): A tweet goes viral — 100,000 likes in 60 seconds.

Trace what happens at each layer:

• Node.js event loop (Chapter 1)
• Redis write-behind counter (Chapter 1, Phase 4)
• BullMQ queue for DB writes (Chapter 4, Phase 4)
• Postgres row contention (Chapter 1, Phase 2 — MVCC and locking)
• Rate limiting on the like endpoint (Chapter 2, Phase 4)

At what point does each layer become the bottleneck? What fails first, what holds up longest?

What's Coming in Chapter 3 of Phase 5

Chapter 1: URL Shortener — read-heavy, simple model, caching solves everything. Chapter 2: Twitter Feed — write-amplification, fan-out, social graph at scale.

Chapter 3 is YouTube — a media system with a completely different bottleneck. The hard problems aren't tweets or feeds — they're:

• Video upload and transcoding (CPU-intensive, not I/O-bound)
• Adaptive bitrate streaming (how your video quality changes with connection speed)
• CDN strategy (a 4K video is 10GB — it cannot come from your origin server)
• View counts at extreme scale (a viral video gets 1M concurrent streams)

The system design reasoning in Chapter 3 will be different from Chapters 1 and 2 precisely because the bottleneck is different. That's the skill: identify the bottleneck first, then design around it.

End of Phase 5, Chapter 2. Estimated reading time: 80-90 minutes. Practice problems: 60-90 minutes. Do not proceed to Chapter 3 until you can answer Problem 4 — the unfollow problem. It has no single correct answer. The quality of your reasoning is what matters. An interviewer asking this question is not looking for "the answer" — they're watching how you think through a tradeoff with no perfect solution. That is the entire skill.