Redis & Job Queues: A Complete Curriculum

From Memory Cells to Production Job Pipelines

Chapter 1: Redis From the Ground Up

This chapter will permanently change how you think about databases. You've been treating Redis as a fast cache — a place to store things that don't need to live in Postgres. That mental model is not wrong, but it's shallow. By the end of this chapter, you'll understand why Redis is fast down to the physical layer, what its data structures actually are in memory, why its single-threaded design is a feature and not a limitation, and why "in-memory" doesn't mean "unreliable." Every design decision in Redis exists for a reason, and once you see the reasons, the entire system becomes obvious.

Part 1: The Physical Reality of Speed

Before you understand Redis, you need to understand why a regular database like Postgres is slow for certain operations — not slow in an abstract sense, but slow because of where data physically lives.

Your computer has several places data can live, each with a dramatically different access cost:

Storage Tier        Latency         Analogy
─────────────────────────────────────────────
CPU Register        0.3 ns          Your hand
L1 Cache            1 ns            Your desk
L2 Cache            4 ns            The room
RAM                 100 ns          Your building
SSD (NVMe)          50,000 ns       A city block away
HDD                 5,000,000 ns    A different country
Network (LAN)       500,000 ns      A neighbouring city

Postgres stores your data on disk — SSD or HDD. When you run a query, Postgres reads 8KB pages from disk into RAM, searches them, and returns results. Even on a fast NVMe SSD, that's 50 microseconds per page read. A table with a million rows might span thousands of pages. Without an index, Postgres reads every single one.

Redis stores data in RAM. RAM access is 100 nanoseconds — 500 times faster than the fastest SSD. When Redis processes a command, it finds the data in memory, operates on it, and returns. No disk read in the hot path. That is the entire physical reason Redis is fast. Everything else is detail on top of this one fact.

But "store things in RAM" is not a novel idea. What makes Redis interesting is what it does with RAM, and how it makes RAM not feel so terrifying to rely on.

Part 2: The Single-Threaded Event Loop — You Already Know This

Here is something that will feel immediately familiar: Redis is single-threaded.

One thread. One operation at a time. Sound familiar? That's your JavaScript event loop — the same architecture, for the same reason.

When Node.js processes a request, it doesn't spawn a new thread for each one. It handles requests one at a time on a single thread, using non-blocking I/O to avoid waiting on slow operations. This works because most operations in JavaScript are I/O-bound — they spend most of their time waiting on the network or disk, not burning CPU.

Redis made the same bet, applied to a database. A Redis command — SET key value, GET key, LPUSH list item — takes microseconds. It's pure memory operations: find the memory address, read or write bytes, return. No disk I/O. No network wait inside the operation. So a single thread can handle tens of thousands of commands per second without ever blocking.

Here's what this single-threaded architecture gives you that is worth far more than raw speed: every Redis command is atomic by definition.

Atomic means indivisible. It means the command either fully completes or doesn't run at all. No other client can see Redis in a half-modified state, because there is no "half-modified state" — only one thing is ever running.

Think about what this means for your SaaS products. In a Postgres-based system, if two requests hit your server simultaneously and both try to update the same record, you need explicit transactions and locks to prevent race conditions. In Redis, you get this for free. Two clients sending commands simultaneously are queued — one runs completely, then the other runs completely. Neither sees the other's partial work.

This is why Redis can be trusted for operations that need to be safe under concurrency: rate limiting, seat holds, distributed locks. Not because Redis has some clever concurrency mechanism — but because it has no concurrency at all inside its core. It serialises everything.

The single-threaded model has one real limitation: a single slow command can block all other commands. Redis has a few commands that can be slow — commands that operate on large collections. This is why Redis has guidelines about not using certain commands (KEYS *, SMEMBERS on huge sets) in production. One slow command blocks the queue behind it. You'll come back to this when you understand BullMQ's queue scanning operations.

Part 3: Data Structures — Not Just "Types," But Physical Choices

Most databases give you tables. Redis gives you data structures — the same data structures you'd find in a CS algorithms textbook, but accessible over a network protocol. Each one is not arbitrary. Each one exists because it enables a class of operations that would be slow or impossible with the others.

Strings

The most basic structure. A key maps to a sequence of bytes. "String" is misleading — it's really a byte array, which means it can hold text, JSON, integers, or binary data. Redis stores small strings inline in the hash table entry to avoid an extra memory allocation. Strings up to 44 bytes get stored directly in the hash table node itself — this is called an embstr encoding. Longer strings get a separate allocation — raw encoding.

Why does encoding matter? Because memory allocations are expensive. The fewer of them Redis does, the faster it runs and the less memory it uses.

What strings give you: O(1) get and set. Atomic increment and decrement (INCR, DECR) — critical for counters and rate limiting. A SET command with expiry baked in (SET key value EX 60) — critical for caches and session tokens. SETNX (set if not exists) — the foundation of distributed locks.

Lists

A doubly linked list. Each node holds a value and pointers to the previous and next nodes. O(1) push and pop from both ends. O(n) access by index (because you have to walk the list).

What lists give you: a queue. Push to the tail (RPUSH), pop from the head (LPOP). Or a stack: push and pop from the same end. The command BLPOP is a blocking pop — the client waits until something is available. This is how naive job queues work: worker sends BLPOP myqueue 0 and sleeps until a job appears.

Lists become a hash-based structure (listpack/ziplist encoding) when they're small — Redis automatically uses a more compact memory representation when the list has fewer than 128 elements and each element is under 64 bytes. Once either threshold is exceeded, it converts to the full linked list. This is an example of Redis's pervasive "use the cheap structure until it hurts" philosophy.

Hashes

A hash map inside a key. Like a JavaScript object nested inside a Redis key. HSET user:1 name "Alice" email "a@b.com" plan "pro" stores a hash at key user:1 with three fields.

What hashes give you: store and retrieve structured data without serialising to JSON. HGET user:1 name returns just the name — you don't have to deserialise the whole object to read one field. HINCRBY user:1 loginCount 1 increments one field atomically. This is far more efficient than GET user:1 → deserialise → increment → serialise → SET user:1.

Small hashes use listpack encoding (a compact array in memory). Large hashes convert to a proper hash table. The engineering principle: linear scan is faster than hash table lookup until you have enough elements that O(n) becomes painful. Redis picks the crossover point empirically.

Sets

An unordered collection of unique strings. Backed by a hash table (so O(1) add, remove, and membership test). Supports set operations: union, intersection, difference.

What sets give you: deduplication and membership testing. "Has this user already received this notification?" — SISMEMBER notified:campaign_42 user_99. O(1). "Which users are in both segment A and segment B?" — SINTER segmentA segmentB. These operations are why recommendation engines and user segmentation tools reach for Redis sets.

Sorted Sets (ZSets)

This is the most important data structure for understanding job queues. A sorted set is a set where every member has a floating-point score. Members are unique; scores can repeat. Redis keeps members sorted by score at all times.

The implementation is a skip list combined with a hash table. The hash table gives O(1) score lookup by member. The skip list gives O(log n) range queries by score. A skip list is a probabilistic data structure — essentially a linked list with express lanes at multiple levels, like a highway interchange. It gives you the range-query ability of a balanced tree without the complexity of rebalancing after insertions.

Skip list concept (simplified):
Level 3: ──────────────────── 50 ──────────────────── 100
Level 2: ──── 20 ──────────── 50 ──── 70 ──────────── 100
Level 1: ── 10 ── 20 ── 30 ── 50 ── 60 ── 70 ── 80 ── 100

To find members with score between 40 and 75, Redis starts at the top level and navigates down — skipping large sections rather than scanning every element.

ZADD queue 1699999999 "job:abc" adds job:abc with Unix timestamp as score. ZRANGEBYSCORE queue 0 1699999999 returns all jobs with score (timestamp) in that range — all jobs due to run before now.

This is exactly how delayed jobs work in BullMQ. The timestamp IS the score. "Run this job at 3pm" means ZADD delayed_queue 1700000000 job_id. At 3pm, BullMQ runs ZRANGEBYSCORE delayed_queue 0 now and finds all jobs that are due. O(log n) for each operation regardless of queue size.

Priority queues work the same way: lower score = higher priority. ZADD priority_queue 1 urgent_job and ZADD priority_queue 100 low_priority_job. Pop the member with the lowest score first.

Streams

Added in Redis 5.0. A stream is an append-only log — like Kafka, but in memory. Each entry has an auto-generated ID (timestamp + sequence number) and a set of key-value fields.

Streams support consumer groups: multiple consumers, each processing different entries, with acknowledgement. If a consumer crashes before acknowledging, Redis tracks the entry as unacknowledged and it can be reclaimed. This is a higher-level reliability mechanism than lists, and it's the direction Redis has been moving toward for robust message processing.

BullMQ currently uses sorted sets and lists rather than streams, but understanding streams tells you where the technology is going and why.

Part 4: Persistence — Why "In-Memory" Doesn't Mean "Fragile"

The obvious worry: if Redis stores data in RAM, a server restart loses everything. This is true and not true depending on your configuration.

Redis has two persistence mechanisms, and understanding them at the physical level matters for how you design systems on top of it.

RDB — Snapshots

Redis periodically forks its process and writes the entire in-memory dataset to disk as a binary snapshot. "Fork" means the operating system creates a copy of the process's memory space — but it uses a technique called copy-on-write. The fork doesn't actually copy all the memory immediately. Both parent and child share the same physical memory pages. Only when one of them modifies a page does the OS create a separate copy of that page.

This means Redis can dump gigabytes of data to disk without pausing for the entire duration. The parent keeps serving requests. The child writes the snapshot. Pages that the parent modifies during the dump get copied — but in a working system, most pages aren't modified during a 2-second snapshot.

The trade-off: if Redis crashes between snapshots, you lose the data since the last snapshot. If your snapshot runs every 5 minutes, you can lose up to 5 minutes of data. For a cache, this is fine. For a job queue, losing 5 minutes of queued jobs is not fine.

AOF — Append-Only File

AOF logs every write command Redis executes, in the order it executed them. On restart, Redis replays the log to reconstruct the in-memory state. It's the same principle as a database write-ahead log.

The fsync policy controls how often the AOF buffer is flushed to disk:

• always: every command is synced. Maximum durability, slowest.
• everysec: synced every second. You can lose at most 1 second of data. The default for most production deployments.
• no: the OS decides when to flush. Fastest, least durable.

With everysec, you have a durable job queue. A Redis crash can only lose the last second of enqueued jobs — and if you're running BullMQ, the jobs were already persisted by the producer before they were considered enqueued, so even that loss can be handled.

AOF files grow forever as commands accumulate. Redis periodically rewrites the AOF — replacing the command history with a minimal set of commands that produces the current state. This is called AOF rewrite and happens automatically based on size thresholds.

Which Do You Use for Job Queues?

For production job queues, the answer is AOF with everysec. RDB alone is not acceptable because the window of data loss is too large. AOF with everysec gives you at-most-1-second loss and reasonable write performance. Many production deployments use both — AOF for durability, RDB as a faster recovery baseline.

Part 5: The Protocol — How Clients Actually Talk to Redis

Redis uses a protocol called RESP (Redis Serialisation Protocol). It's worth understanding once, not because you'll implement it, but because it'll remove the mystery of what a Redis "command" actually is over the wire.

RESP is a line-based text protocol. A command is sent as an array of bulk strings:

*3\r\n           ← array of 3 elements
$3\r\n           ← first element is 3 bytes
SET\r\n
$5\r\n           ← second element is 5 bytes
mykey\r\n
$7\r\n           ← third element is 7 bytes
myvalue\r\n

Redis responds:

+OK\r\n          ← simple string response

That's it. The protocol is intentionally simple so it's fast to parse and easy to implement in any language. Redis clients in Node.js, Python, Go — they all send exactly this format over a TCP connection.

Redis 3.0 introduced pipelining: sending multiple commands without waiting for each response. The client sends 100 commands in one TCP packet, then reads 100 responses. This dramatically reduces round-trip latency when you need to execute many commands together. BullMQ uses pipelining extensively when processing jobs.

Multi/Exec gives you transactions: MULTI starts a block, you send commands, EXEC runs them all atomically. No other client can see intermediate state. But Redis transactions don't roll back on error — if command 3 fails, commands 1 and 2 already ran. This is different from SQL transactions. Redis's atomic unit is the command, not the transaction block.

Lua scripting is the real answer to complex atomic operations. EVAL runs a Lua script server-side, atomically. Since Redis is single-threaded, the script runs to completion without interruption. BullMQ's most critical operations — moving a job from waiting to active, checking a job's state and updating it — are implemented as Lua scripts for exactly this reason. We'll see this concretely in Chapter 3.

Vocabulary Map — Chapter 1

Practice Problems — Chapter 1

Problem 1 — Failure Analysis

You're running Redis for your SaaS as a pure cache (no persistence enabled). Your EC2 instance crashes at 3am. When it restarts, Redis starts empty. Meanwhile, you have 2,000 users whose session tokens were stored in Redis — they all get logged out. Walk through: (a) which Redis persistence configuration would have prevented this, (b) what the maximum data loss would be with that configuration, and (c) what the trade-off is in terms of write performance. Is there a configuration that gives you zero data loss? What does it cost?

Problem 2 — Data Structure Selection

For each of the following use cases, choose the Redis data structure and explain why that structure's physical properties make it the right choice. A wrong answer with the right reasoning is more valuable than a right answer with no reasoning.

• Rate limiting: "this IP address can make at most 100 requests per minute"
• Session storage: store userId, plan, and email for logged-in users
• Leaderboard: top 100 users by score, updated in real-time, needs rank queries
• "Has this user seen this onboarding tooltip?": fast membership check per user per tooltip ID
• Job queue that runs jobs in order of submission

Problem 3 — The Atomic Guarantee

A teammate says: "We can use Redis's INCR command to implement rate limiting — just increment a counter per IP per minute. If the counter exceeds 100, reject the request." You point out there's a potential bug. What is it? (Hint: the INCR is atomic, but what about the surrounding logic?) Write the correct Redis command sequence that handles this atomically. Then explain: why can't this be solved with Redis MULTI/EXEC transactions, and what can solve it?

Problem 4 — Sorted Sets at Scale

You're building a delayed notification system. Users schedule notifications up to 30 days in advance. At peak, there are 10 million scheduled notifications in the sorted set. Every second, a worker runs ZRANGEBYSCORE notifications 0 <now> to find due notifications and processes them.

Describe what happens to this system at 100,000 notifications due at the exact same second (a scheduled newsletter blast). What's the time complexity of the scan? What's the problem with processing all of them in the same thread? Design a solution that keeps the sorted set but handles the burst without blocking Redis.

Problem 5 — Go Deeper (connects to JS event loop)

Redis's single-threaded event loop and Node.js's single-threaded event loop are architecturally similar — both use I/O multiplexing (epoll/kqueue) to handle many concurrent connections on one thread without blocking. But there's a critical difference in what "slow operation" means in each context.

In Node.js, the danger is CPU-intensive synchronous code blocking the event loop — the event loop can't yield to other callbacks while a for loop runs for 5 seconds. In Redis, the equivalent danger is a slow command.

Find three Redis commands that are O(n) or worse in their time complexity (where n is the size of the data structure). For each one, describe the production scenario where this becomes a real problem. Then explain: Redis 6.0 introduced threaded I/O for reading and writing network data, while keeping command processing single-threaded. Why does this help, and what problem does it not solve?

What's Coming Next

You now know what Redis is at the layer below the abstraction: a single-threaded, in-memory, data-structure server whose speed comes from RAM access times and whose reliability comes from AOF persistence. The next chapter builds the job queue on top of this foundation — from first principles, without any library. You'll write a queue using raw Redis commands, discover exactly where it breaks, and understand why each of BullMQ's design decisions exists as a direct response to a specific failure mode. By the time you see BullMQ's actual implementation in Chapter 3, every choice will be obvious.

Chapter 2: Job Queues From First Principles

Your instinct when you first needed to run slow work asynchronously was probably to call the function and not await it, or to use setTimeout. These work until they don't — and the moment they don't, they fail silently, permanently, and in production. This chapter explains why that instinct breaks, builds a real job queue from raw Redis commands, and breaks it in every interesting way so you understand exactly what BullMQ is solving. Every production failure mode has a name and a mechanism. After this chapter, you'll be able to diagnose them.

Part 1: The Physical Problem That Queues Solve

When a user uploads a file to your SaaS and you need to process it — generate a PDF, run OCR, resize images, call an AI API — you have a fundamental problem with the HTTP request model.

An HTTP request has a timeout. The browser, the load balancer, the CDN — all of them will cut the connection after some seconds if there's no response. A common default is 30 seconds. Your AI processing call takes 45 seconds. The HTTP request dies. The user gets an error. But the processing might still be running on your server — except now nobody is waiting for the result, and the half-processed state is a mystery.

Even if you could keep the connection alive, it's the wrong model. HTTP is synchronous: the client waits, doing nothing, while your server does work. For 45 seconds, that client connection holds a socket open, a thread (or event loop tick) blocked, and a user staring at a spinner. If 1,000 users do this simultaneously, you have 1,000 connections open for 45 seconds each. Your server is busy doing slow work, not accepting new connections. New users get 504s.

The solution is to decouple receipt from processing. When the user uploads the file, your server does three things:

1. Saves the file to storage (fast — seconds).
2. Creates a record in the database: "processing job pending for file X".
3. Returns 200 immediately: "Your file is being processed. We'll notify you."

The actual processing happens somewhere else, started by something else, at some point after the request is done. That "somewhere else" is a worker. The mechanism that connects the web server to the worker is a queue.

┌──────────────┐         ┌─────────────────┐         ┌──────────────┐
│  Web Server  │ ──job──▶│   Job Queue     │ ──job──▶│    Worker    │
│  (responds   │         │   (Redis)       │         │ (processes   │
│   in <1s)    │         │                 │         │  the job)    │
└──────────────┘         └─────────────────┘         └──────────────┘
       ↑                                                     │
       └─────────────── result / notification ───────────────┘

The web server and the worker are now independent processes. The queue is the durable buffer between them. The web server can be fast because it doesn't do slow work. The worker can be slow because nobody is waiting on it. And if the worker crashes, the job is still in the queue — not lost inside a dead HTTP connection.

This is the physical architecture that a job queue enables. Everything else is details about making it reliable.

Part 2: Building a Queue From Raw Redis — The Naive Version

The simplest possible job queue using Redis is a list. You push jobs onto one end; workers pop from the other.

Producer:   RPUSH jobs '{"type":"process_upload","fileId":"abc123"}'
Worker:     BLPOP jobs 0       ← blocks until a job appears, returns it

RPUSH adds to the right (tail) of the list. BLPOP pops from the left (head) and blocks if the list is empty. This gives you FIFO ordering — first in, first out. The worker sleeps efficiently (the OS wakes it only when data arrives) rather than polling in a loop.

This works. For a toy application. Let's break it in every way it can break.

Part 3: Breaking the Naive Queue — Five Failure Modes

Failure Mode 1: The Worker Crashes Mid-Job

A worker pops a job with BLPOP. The job is now gone from the queue. The worker starts processing — calls the AI API, halfway through, the process crashes. Power failure, OOM kill, segfault in a native module, deployment restart. The job is gone. Not in the queue. Not anywhere. Permanently lost. The user's file is never processed.

This is called at-most-once delivery — the job is delivered to a worker at most once, whether or not it succeeded. For anything important, this is unacceptable.

The fix requires a two-step approach: don't remove the job from the queue until the worker signals completion. Instead, atomically move the job from a "waiting" list to a "processing" list when the worker picks it up. When the worker finishes successfully, remove it from the processing list. If the worker crashes, the job is still in the processing list — it can be reclaimed.

BRPOPLPUSH waiting_queue processing_queue 0

BRPOPLPUSH atomically pops from the right of one list and pushes to the left of another — one atomic Redis operation, so the job is never in a state where it's in neither list. Now if the worker crashes, the job is in processing_queue. A recovery process (the "stalled job reaper") can periodically scan processing_queue, find jobs that have been there too long, and re-enqueue them.

This gives you at-least-once delivery — the job will be delivered at least once, possibly more if the worker crashes after partially completing the job. For idempotent operations (ones where running twice has the same effect as running once), this is acceptable.

Note: BRPOPLPUSH is deprecated in Redis 6.2, replaced by LMOVE with BLOCK option. BullMQ uses LMOVE. The concept is identical.

Failure Mode 2: One Worker, One Job at a Time

BLPOP is blocking — while one worker waits, it processes one job at a time. If you have 1,000 jobs and each takes 10 seconds, processing all of them takes 10,000 seconds. You need multiple workers.

This is straightforward: run multiple worker processes, each with its own BLPOP connection. Redis handles this correctly — when multiple clients are all blocking on BLPOP jobs, and a job arrives, Redis wakes exactly one of them (the one that's been waiting longest). This is fair distribution.

But now the "processing" list approach from Failure Mode 1 has a problem: multiple workers pop different jobs into the same processing_queue. The stalled job reaper needs to know which job belongs to which worker, and how long each has been there, to determine if a job is actually stalled vs. still running.

You need each job to carry a timestamp of when it was picked up, and the stalled job reaper needs to compare now - pickup_time > max_job_duration to find actual stalls.

Failure Mode 3: All Retries Are the Same

A job fails. The worker throws an error. What happens? In the naive implementation, nothing — the job was already popped from the queue. With the BRPOPLPUSH approach, you need to explicitly re-enqueue it.

But naive re-enqueueing puts it at the back of the queue immediately. If the reason it failed is a temporary downstream outage — the AI API returned 503 — retrying immediately just hits the same 503. You need exponential backoff: wait 1 second, then 2 seconds, then 4 seconds, then 8 seconds... up to some maximum, then give up.

Immediate re-enqueueing is also unfair to other jobs: one stuck job that keeps failing immediately gets retried over and over, consuming worker capacity and possibly stalling the whole queue.

Implementing delayed retry requires the sorted set structure from Chapter 1. When a job fails and needs to be retried in 30 seconds:

ZADD retry_queue <now + 30000> job_data

A separate process (or the worker itself, periodically) runs:

ZRANGEBYSCORE retry_queue 0 <now>

and moves due-to-retry jobs back to the waiting queue. This is not a simple list pop — it requires reading from the sorted set and pushing to the waiting list atomically. Otherwise, a crash between the two steps loses the job. This is where Lua scripting becomes essential.

Failure Mode 4: The Queue Grows Without Bound

The producer is fast; the worker is slow. The producer enqueues 1,000 jobs per minute; each worker processes 100 per minute. With 5 workers, you process 500 per minute. The queue grows by 500 jobs every minute. In an hour: 30,000 jobs backed up. In a day: 720,000 jobs. Redis's RAM fills up. Redis OOMs. Everything stops.

This is called queue saturation or backpressure failure. The solution is backpressure: the producer should slow down or fail when the queue is too deep, rather than producing indefinitely.

The naive queue has no mechanism for this. You need to add a check before enqueueing: if LLEN jobs > 10000, reject the job with an error rather than adding to an already-overloaded queue. This is better than silently accepting jobs that will never be processed.

BullMQ exposes queue size limits for exactly this reason.

Failure Mode 5: Job Results Go Nowhere

The worker processes the job and produces a result — a processed PDF, an OCR transcript, a resized image. Where does the result go?

The naive queue doesn't have an answer. The worker is a separate process from the web server that originally received the request. The user's HTTP connection is long gone. You need a way for the worker to store the result and for the user to retrieve it — either by polling a status endpoint, or by receiving a notification (webhook, WebSocket, email).

This means every job needs: a job ID (so results can be stored and retrieved by ID), a status field (pending, active, completed, failed), a result field (written by the worker on completion), and a way to notify interested parties of status changes.

Part 4: The Queue State Machine

A production job queue treats every job as a state machine. The job transitions through states, and each transition is atomic:

                    ┌─────────┐
                    │ WAITING │  ← job is enqueued, waiting for a worker
                    └────┬────┘
                         │ worker picks it up (atomic move)
                         ▼
                    ┌─────────┐
                    │ ACTIVE  │  ← worker is processing it
                    └────┬────┘
              ┌──────────┴──────────┐
              │ success             │ failure
              ▼                     ▼
        ┌──────────┐         ┌──────────────┐
        │COMPLETED │         │    FAILED    │  ← max retries exceeded
        └──────────┘         └──────────────┘
                                    │
                         retry attempts remain
                                    │
                                    ▼
                             ┌────────────┐
                             │  DELAYED   │  ← waiting for backoff timer
                             └─────┬──────┘
                                   │ timer fires
                                   ▼
                              ┌─────────┐
                              │ WAITING │  ← back in the queue
                              └─────────┘

There's also a special state for when a worker crashes mid-job:

ACTIVE → (worker crashes, no acknowledgement) → STALLED
STALLED → (stall reaper detects it) → WAITING (retry) or FAILED (too many stalls)

Each of these states corresponds to a physical location in Redis:

WAITING  → sorted set (with timestamp as score, for ordering)
ACTIVE   → hash (jobId → job data + pickup timestamp)
COMPLETED → sorted set (capped to last N jobs for inspection)
FAILED   → sorted set (capped to last N failures for inspection)
DELAYED  → sorted set (with future timestamp as score)

The transitions between these states must be atomic — which is why BullMQ implements them as Lua scripts, not as sequences of Redis commands.

Part 5: The Stalled Job Problem — The Hardest Part

The hardest problem in job queue design is the stalled job: a worker picked up a job, started processing, and then fell silent. Is it still running, just slowly? Or did it crash?

You can't know from outside. From Redis's perspective, a job that's been active for 2 minutes looks identical whether the worker is still running or has been dead for 90 seconds.

The solution is a heartbeat protocol combined with a stall detector.

Heartbeat: While processing a job, the worker periodically updates a timestamp in Redis: "I'm still alive, last checked in at time T." The interval must be shorter than the stall detection window. If the worker checks in every 5 seconds, and the stall detector runs every 30 seconds, a crashed worker will be detected within 35 seconds.

Stall detector: Periodically scans all active jobs. For each one, checks: has the worker checked in within the last 30 seconds? If not, the job is stalled. Move it back to waiting (if retry attempts remain) or to failed (if not).

The stall detector must itself be atomic — it needs to check the timestamp and move the job in one operation, or two simultaneous stall detectors could both decide to re-queue the same job, leading to double processing.

This is the fundamental tension in at-least-once queues: you will sometimes process a job twice (once by the stalled worker that was actually still running but slow, and once by the new worker that picked up the re-queued job). Your job handlers must be designed with this in mind — they must be idempotent.

Idempotent means: processing the job twice produces the same result as processing it once. "Send a password reset email" is not idempotent — processing it twice sends two emails. You make it idempotent by tracking in your database whether the email was already sent: "if this reset token already has a sent_at timestamp, skip the send."

Vocabulary Map — Chapter 2

Practice Problems — Chapter 2

Problem 1 — Failure Analysis: The Double-Processing Bug

You're running a queue that sends invoices. The job handler: look up the order, generate PDF, email PDF to customer, mark order as invoiced. The stall timeout is 30 seconds. A worker picks up a job and spends 25 seconds generating the PDF. At second 29, the stall detector fires, decides the worker is stalled, and re-queues the job. At second 30, the original worker finishes and emails the PDF.

A new worker picks up the re-queued job and emails the PDF again. The customer receives two invoices.

Identify: (a) the sequence of events that caused this, (b) the specific configuration parameter that made it likely, (c) how to fix it without removing the stall detector, and (d) how to make the job handler idempotent so that even if double-processing happens, no harm occurs.

Problem 2 — Design: A Queue With Priorities

You have two classes of jobs: user-triggered jobs (a user clicks "export my data" — they're waiting) and system jobs (nightly report generation — nobody is watching). User jobs must always be processed before system jobs, even if 1,000 system jobs are already in the queue.

Design this using raw Redis data structures from Chapter 1. What structure holds each type? What does the worker's polling logic look like? What happens if user jobs arrive faster than workers can process them — does the system job queue stall forever?

Problem 3 — Synthesis: Backpressure + Retry + Dead Letter

You're building a webhook delivery system for your SaaS. Your app sends webhooks to customer endpoints when events occur. The customer's endpoint might be down (503) — you need to retry. The customer's endpoint might be permanently gone — you need to give up eventually. Your webhook event rate can spike (product launch) — you need to handle bursts without dropping events.

Design the complete queue architecture: data structures, job states, retry policy (what backoff formula?), when to move to DLQ, how to handle backpressure from the producer side. Use only concepts from Chapters 1 and 2.

Problem 4 — The Exactly-Once Illusion

A senior engineer tells you: "For payment processing, we need exactly-once delivery. At-least-once is unacceptable — we can't charge a customer twice." You explain that exactly-once delivery in a distributed system is fundamentally impossible without specific conditions. What are those conditions? Can a Redis-based job queue provide exactly-once delivery for payment jobs? If not, what is the correct architecture for processing payments idempotently? Be specific about what goes in the queue, what goes in the database, and what check the job handler performs.

Problem 5 — Go Deeper (connects to Chapter 1: sorted sets)

BullMQ uses a sorted set for delayed jobs, with the execution timestamp as the score. A worker periodically runs ZRANGEBYSCORE delayed 0 <now> to find due jobs, then moves them to the waiting queue one at a time.

There's a subtle race condition in this design when you run multiple workers. Describe the race. (Hint: what happens between the ZRANGEBYSCORE read and the ZREM/LPUSH move?) How does BullMQ solve this? What Redis feature makes the solution atomic? Now consider: if you have 50,000 delayed jobs all due at exactly the same second (a scheduled blast), what happens to Redis during that second? What's the time complexity of the scan, and what's the impact on other Redis operations during that second?

What's Coming Next

You've built a queue from scratch and broken it in every important way. You understand why each failure mode exists at the mechanical level. Now we look at BullMQ — not as a black box library, but as a set of engineering decisions, each one a direct response to a failure mode you just studied. We'll trace exactly which Redis commands BullMQ executes for each operation, read the actual Lua scripts that handle state transitions, and understand every configuration option from first principles. By the end, you'll be able to look at a BullMQ production failure, read the Redis state directly, and diagnose it without Stack Overflow.

Chapter 3: BullMQ Internals and Production Mastery

You know why queues exist. You know how to build one from Redis and where it breaks. Now we look at BullMQ — not the documentation, but the actual decisions made by engineers who've run job queues at scale and been burned by every failure mode in Chapter 2. Every API, every configuration option, every internal mechanism is a response to something that went wrong in production. Read this chapter that way and nothing will feel arbitrary.

Part 1: The Data Model — What BullMQ Actually Stores in Redis

When you create a BullMQ queue named "email", BullMQ creates a set of Redis keys with a predictable naming pattern. Understanding these keys is the foundation of being able to inspect and debug a production queue directly.

bull:{email}:1               ← Hash: the job data for job ID 1
bull:{email}:waiting         ← List: job IDs waiting for a worker
bull:{email}:active          ← List: job IDs currently being processed
bull:{email}:completed       ← Sorted set: completed job IDs (score = completion timestamp)
bull:{email}:failed          ← Sorted set: failed job IDs (score = failure timestamp)
bull:{email}:delayed         ← Sorted set: delayed job IDs (score = execution timestamp)
bull:{email}:paused          ← List: job IDs in a paused queue
bull:{email}:priority        ← Sorted set: job IDs with priorities (score = priority value)
bull:{email}:stalled-check   ← Key used for distributed stall detection locking
bull:{email}:meta            ← Hash: queue metadata (paused state, job counts)
bull:{email}:id              ← String: auto-incrementing job ID counter

The curly braces around email are Redis Cluster hash tags — they ensure all keys for the same queue land on the same Redis node in a cluster. Without them, the atomic Lua scripts that operate on multiple keys would fail because the keys might live on different nodes.

The job hash: When you enqueue a job, BullMQ stores the job data as a hash at bull:{email}:<id>:

bull:{email}:1
  id:          "1"
  name:        "sendWelcomeEmail"
  data:        '{"userId":"user_abc","email":"a@b.com"}'
  opts:        '{"attempts":3,"backoff":{"type":"exponential","delay":1000}}'
  timestamp:   "1700000000000"
  delay:       "0"
  priority:    "0"
  attemptsMade: "0"
  stacktrace:  "[]"
  returnvalue: null
  processedOn: null
  finishedOn:  null

The job's payload is data. The job's configuration is opts. Everything else is runtime state. Notice that attemptsMade, stacktrace, processedOn, and finishedOn are updated by BullMQ as the job progresses. You can inspect any job's state at any time:

redis-cli HGETALL bull:email:42

This is your primary debugging tool for production. If a job seems stuck, get its data directly from Redis.

Part 2: The Enqueue Operation — What queue.add() Does

When you call:

await queue.add("sendWelcomeEmail", { userId: "user_abc", email: "a@b.com" });

BullMQ executes a Lua script. Not a sequence of commands — a single atomic Lua script that does all of the following:

1. Increments the job ID counter: INCR bull:{email}:id → returns, say, 42
2. Stores the job data as a hash: HSET bull:{email}:42 id 42 name sendWelcomeEmail data {...} ...
3. Checks if the queue is paused: if paused, pushes ID to bull:{email}:paused list
4. Checks if the job has a delay: if delay > 0, adds to bull:{email}:delayed sorted set with future timestamp as score
5. Checks if the job has a priority: if priority > 0, adds to bull:{email}:priority sorted set with priority as score
6. Otherwise: pushes the job ID to the left of bull:{email}:waiting list: LPUSH bull:{email}:waiting 42
7. Publishes a notification: PUBLISH bull:{email}:events '{"event":"waiting","jobId":"42"}' — so workers that are idle get woken up

All of this happens in one Lua script, atomically. No other client can see the queue in an intermediate state between steps 2 and 6. The job either fully exists in the waiting list with all its data, or it doesn't exist at all.

Why does it publish a notification? Because BLPOP-based workers are efficient — they sleep until notified. But BullMQ workers don't use BLPOP directly (we'll see why shortly). Instead, they subscribe to events via Redis pub/sub and wake up to process when notified. This decoupling allows a single event to wake multiple workers, which can then race for the job.

Part 3: The Process Loop — What a Worker Actually Does

A BullMQ worker runs an event loop. On each iteration:

1. Try to move a job from waiting to active — the critical atomic step.
2. If a job was moved, process it.
3. On completion (success or failure), update job state.
4. Repeat.

Step 1 is implemented as a Lua script called moveToActive. Simplified logic:

-- Get the next job ID from the waiting list
local jobId = redis.call('RPOP', KEYS[1])  -- KEYS[1] = waiting list

if jobId then
  -- Move it to the active list
  redis.call('LPUSH', KEYS[2], jobId)  -- KEYS[2] = active list

  -- Record when it was picked up (for stall detection)
  redis.call('HSET', KEYS[3] .. jobId, 'processedOn', ARGV[1])  -- ARGV[1] = now

  -- Return the job data
  return redis.call('HGETALL', KEYS[3] .. jobId)
end

return nil  -- no jobs available

This Lua script runs atomically in Redis. Either the job is in waiting and moves to active, or nothing happens. There's no window where the job is in neither list. This is the atomic handoff that solves Failure Mode 1 from Chapter 2.

Why RPOP from the waiting list (not LPOP)? Because jobs are pushed with LPUSH (to the front) and the most recently added job is at the front of the list. Actually — BullMQ pushes with LPUSH and pops with RPOP to get FIFO ordering. First in (earliest LPUSH) ends up at the right side (RPOP pops from right). FIFO is the expected behaviour for a queue.

Priority queues: If the queue has prioritised jobs, moveToActive first checks the bull:{email}:priority sorted set. ZPOPMIN gets the member with the lowest score (highest priority). If there are prioritised jobs, they're processed before the regular waiting list. If not, fall through to the normal waiting list.

Part 4: Stall Detection — The Distributed Heartbeat

BullMQ's stall detection runs periodically (configurable, default every 5 seconds per worker) and implements the heartbeat + stall reaper pattern from Chapter 2.

The mechanism is a separate sorted set bull:{email}:stalled where active job IDs are stored with the timestamp they were last seen. Periodically, the moveToActive Lua script also updates this timestamp for the current job. A separate process (the stall checker) runs checkStalledJobs on an interval:

-- Find jobs that haven't been updated recently
local stalled = redis.call('ZRANGEBYSCORE', stalledKey, 0, now - stalledInterval)

for _, jobId in ipairs(stalled) do
  -- Check retry attempts
  local attempts = redis.call('HGET', prefix .. jobId, 'attemptsMade')
  local maxAttempts = -- from job opts

  if tonumber(attempts) < maxAttempts then
    -- Re-queue: move from active to waiting
    redis.call('LREM', activeKey, 1, jobId)
    redis.call('LPUSH', waitingKey, jobId)
    redis.call('HINCRBY', prefix .. jobId, 'attemptsMade', 1)
  else
    -- Exceeded retries: move to failed
    redis.call('LREM', activeKey, 1, jobId)
    redis.call('ZADD', failedKey, now, jobId)
    redis.call('HSET', prefix .. jobId, 'finishedOn', now)
  end

  -- Remove from stalled tracking
  redis.call('ZREM', stalledKey, jobId)
end

Again: all of this is a single atomic Lua script. No two workers can both decide to re-queue the same stalled job because the Lua script removes the job from stalled before returning — the next worker that runs the script won't see the same job.

The stall interval configuration (stalledInterval) is critical. Set it too low (1 second) and slow legitimate jobs get falsely detected as stalled and re-queued. Set it too high (5 minutes) and crashed workers hold jobs hostage for 5 minutes before recovery. The right value is: (maximum legitimate job duration) + (some buffer for slowness). If your jobs take at most 30 seconds, set stalledInterval to 45 seconds.

If a job legitimately needs 10 minutes (video processing, large AI inference), you have two options: set stalledInterval to 12 minutes (everyone waits 12 minutes for crashed worker recovery), or have the worker actively extend its lock — call job.extendLock(duration) periodically to tell Redis "I'm still alive, give me more time."

Part 5: Delayed Jobs — The Sorted Set Promotion Loop

When you enqueue a delayed job:

await queue.add("sendReminder", data, { delay: 3600000 }); // 1 hour from now

BullMQ computes now + delay = future_timestamp and stores the job ID in bull:{email}:delayed sorted set with that timestamp as the score.

The promotion loop runs on each worker and on a dedicated QueueScheduler (now built into BullMQ v2 Worker). It executes moveStalledJobsToWait which includes:

-- Find delayed jobs whose time has come
local now = tonumber(ARGV[1])
local jobs = redis.call('ZRANGEBYSCORE', delayedKey, 0, now, 'LIMIT', 0, ARGV[2])

for _, jobId in ipairs(jobs) do
  redis.call('ZREM', delayedKey, jobId)
  redis.call('LPUSH', waitingKey, jobId)
end

ARGV[2] is the batch size — BullMQ only promotes a limited number of delayed jobs per tick to avoid blocking Redis for too long (remember: slow Lua script = blocks all other commands). The default is 1,000 per tick.

This runs every 1 second by default. So a job with delay: 5000 will actually run somewhere between 5 and 6 seconds after enqueue — not exactly at 5 seconds. The precision of delayed jobs in BullMQ is ±1 polling interval.

Part 6: Concurrency — How Workers Process Multiple Jobs Simultaneously

A single BullMQ Worker instance can process multiple jobs concurrently. This is controlled by the concurrency option:

const worker = new Worker("email", processJob, {
  connection: redisConnection,
  concurrency: 5,  // process up to 5 jobs simultaneously
});

Under the hood, this means the worker runs the moveToActive loop up to 5 times simultaneously. Each concurrent job runs your processJob function as a separate async execution. Since JavaScript is single-threaded but non-blocking, 5 concurrent jobs means 5 async operations in flight, sharing the single event loop — appropriate if your job handlers are I/O-bound (network calls, database queries).

If your jobs are CPU-intensive (image processing, PDF generation), concurrency on a single Worker instance buys you nothing because the CPU is the bottleneck, not I/O. For CPU-intensive work, you need multiple worker processes — either multiple Node.js processes or worker threads with their own event loop. BullMQ supports worker threads natively.

The concurrency limit in Redis: When a worker tries to take more than concurrency jobs, it simply doesn't call moveToActive again until one of its current jobs completes. The "concurrency" is enforced locally in the worker process — Redis doesn't know or care how many jobs each worker has taken. What Redis sees is: N things in the active list. Whether they're all in one worker or spread across 10 workers is irrelevant to Redis.

This has an implication: if you have concurrency: 10 and 10 workers, Redis might show 100 jobs in the active list simultaneously. If they all stall, 100 jobs need to be re-queued by the stall detector. This is fine — just know what you're monitoring.

Part 7: Rate Limiting — Throttling Job Processing

Sometimes you need to limit how fast jobs process — not because of worker capacity, but because the downstream system has a rate limit. Your AI provider allows 100 API calls per minute. You have 10 workers, each with concurrency: 5 — that's 50 simultaneous jobs, potentially 50 API calls per second. You'll get 429s.

BullMQ's rate limiter uses a sliding window implemented with Redis:

const worker = new Worker("ai-jobs", processJob, {
  connection,
  limiter: {
    max: 100,     // max 100 jobs per duration
    duration: 60000, // per 60 seconds
  },
});

Internally, BullMQ stores a counter in Redis with a 60-second expiry. Before moving a job to active, the Lua script checks: has the counter exceeded max? If yes, the job stays in waiting and the worker sleeps. If no, increment the counter and proceed.

The counter key is bull:{email}:limiter. The Lua script does an INCR with EXPIRE to create a sliding window. This counter is shared across all worker instances — 10 workers all check and update the same counter atomically. No worker can exceed the rate limit even in a distributed setup, because the check and increment are in one Lua script.

Part 8: Job Events and the Event Bus

BullMQ exposes job lifecycle events. Every time a job transitions state, BullMQ publishes an event to Redis pub/sub:

bull:{email}:events  ← pub/sub channel

Event messages look like:

{"event":"completed","jobId":"42","returnvalue":"...","prev":"active"}
{"event":"failed","jobId":"43","failedReason":"Error: timeout","prev":"active"}
{"event":"progress","jobId":"44","data":50,"prev":"active"}

You subscribe to these with QueueEvents:

const queueEvents = new QueueEvents("email", { connection });

queueEvents.on("completed", ({ jobId, returnvalue }) => {
  console.log(`Job ${jobId} completed with:`, returnvalue);
});

queueEvents.on("failed", ({ jobId, failedReason }) => {
  console.log(`Job ${jobId} failed:`, failedReason);
});

This is the Observer pattern from Chapter 4 of your LLD curriculum, implemented using Redis pub/sub as the event bus. The worker is the Subject — it emits events. The QueueEvents listener is the Observer. They're completely decoupled: the worker doesn't know who's listening, and the listener doesn't affect the worker.

job.updateProgress(50) publishes a progress event. Your web server can subscribe to this event and push it to the user's browser via WebSocket — giving the user a real-time progress bar on their long-running job, without the worker ever needing to know about WebSockets.

The critical limitation: Redis pub/sub is fire-and-forget. If no subscriber is listening when an event is published, the event is lost. QueueEvents is not a reliable event log — it's a real-time notification channel. If you need to know that a job completed even if your web server was restarted, read the job's state directly from Redis, not from pub/sub.

Part 9: The Flow — Parent and Child Jobs

Real-world tasks are often a pipeline: step 1 must complete before step 2 starts, which must complete before step 3. Or: one parent job fans out to many parallel child jobs, and you need to know when all children have completed.

BullMQ Flows handle this natively. A Flow is a tree of jobs where:

• Parent job waits for all its children to complete before it can start
• Children can have their own children
• Failure of any child fails the parent (configurable)

const flow = new FlowProducer({ connection });

await flow.add({
  name: "processOrder",
  queueName: "orders",
  data: { orderId: "order_123" },
  children: [
    { name: "validateInventory", queueName: "inventory", data: { orderId: "order_123" } },
    { name: "chargePayment",     queueName: "payments",  data: { orderId: "order_123" } },
    { name: "reserveShipping",   queueName: "shipping",  data: { orderId: "order_123" } },
  ],
});

BullMQ enqueues all children immediately (they can process in parallel). The parent goes into a waiting-children state — not processing, just waiting. As each child completes, BullMQ decrements a counter on the parent. When the counter reaches zero (all children done), the parent moves to waiting and a worker picks it up.

This is implemented with an additional set per parent job: bull:{orders}:dependencies:{parentId} — a set of child job IDs that haven't completed yet. Each child completion atomically removes its ID from this set. When the set is empty, the parent is promoted.

Part 10: Production Operations — Monitoring, Scaling, and Recovery

Monitoring What Matters

The key metrics for a healthy queue:

Waiting count:    how many jobs are pending — spike indicates producer outburst or worker failure
Active count:     how many jobs are in-flight — should be ≤ (concurrency × worker count)
Failed count:     how many jobs have exhausted retries — should be zero or have an alert
Delayed count:    how many jobs are scheduled for the future — expected to be non-zero
Completed count:  total completions — use rate (completions/sec) not total
Processing time:  p50, p95, p99 of job duration — spike indicates downstream slowness

You can get all of these from BullMQ:

const counts = await queue.getJobCounts(
  "waiting", "active", "completed", "failed", "delayed"
);

Or directly from Redis:

LLEN bull:email:waiting     ← waiting count
LLEN bull:email:active      ← active count  
ZCARD bull:email:failed     ← failed count
ZCARD bull:email:delayed    ← delayed count

Set up an alert on failed count — if it's non-zero and growing, something is wrong with your job handlers. Set up an alert on waiting count exceeding a threshold — if jobs pile up, you need more workers.

Scaling Workers

Adding more workers is a matter of running more Node.js processes pointing at the same Redis instance. BullMQ handles the coordination — workers race atomically for jobs via the Lua scripts. There's no configuration change needed in Redis or BullMQ to add workers. The moveToActive Lua script is the serialisation point — even with 100 workers running simultaneously, each job is picked up by exactly one worker.

The practical limit on worker count is determined by: the number of Redis connections (each worker holds at least 2 connections — one for commands, one for blocking operations), the Redis server's connection limit (default 10,000), and the Redis server's CPU (the Lua scripts run on Redis's single thread — 100 workers all calling moveToActive simultaneously serialise through this).

On a single Redis instance, you can typically run hundreds of workers efficiently. Beyond that, you need Redis Cluster (sharding jobs across multiple nodes) or separate Redis instances per queue.

Handling the DLQ — Failed Jobs

Jobs in the failed set are your dead letter queue. You need a strategy for them:

// Inspect failed jobs
const failedJobs = await queue.getFailed(0, 99); // first 100 failed jobs

for (const job of failedJobs) {
  console.log(job.id, job.failedReason, job.stacktrace);
  
  // If the failure was a transient issue, retry manually
  await job.retry();
  
  // If it's permanently broken, discard it
  await job.remove();
}

For production, build a simple admin UI that lists failed jobs with their reason, data, and stacktrace. This is called a DLQ viewer. It's the difference between "we don't know why 200 jobs failed last night" and "here's the exact error and payload for every failed job."

Draining and Pausing

If you need to deploy a new version of your worker without dropping in-flight jobs:

// Signal: stop picking up new jobs, finish current ones
await worker.pause();

// Wait for all current jobs to finish
await worker.close();

// Now deploy the new version

For emergency stops (bad deploy, bug in job handler causing damage):

// Immediately stop all workers
await queue.pause();       // pause the queue — no new jobs picked up by any worker
await worker.close(true);  // force close — abandon current jobs (they'll be re-queued by stall detector)

Part 11: Repeatable Jobs — Cron in Redis

BullMQ supports repeatable jobs — jobs that run on a schedule, like cron:

await queue.add(
  "generateDailyReport",
  { reportType: "daily" },
  {
    repeat: {
      pattern: "0 9 * * *",  // 9am every day, cron syntax
      // or:
      every: 3600000,         // every hour in milliseconds
    },
  }
);

BullMQ stores the repeat definition in a separate sorted set: bull:{email}:repeat. The scheduler (built into the Worker) periodically checks this set, finds schedules that are due, creates new job instances in the waiting queue, and computes the next run time.

This is cron-in-Redis. The advantage over OS cron: the schedule is stored in Redis, shared across all workers. Only one worker picks up each execution (the Lua script atomicity guarantee). If a worker running the scheduler crashes, another worker becomes the scheduler. No single point of failure, no external cron service to manage.

Vocabulary Map — Chapter 3

Practice Problems — Chapter 3

Problem 1 — Failure Analysis: The Missing Jobs

You're running BullMQ in production. Yesterday, 47 jobs that were enqueued never appeared in the completed or failed lists. The workers logged no errors. The queue's waiting and active counts are both 0. Where did the jobs go? Walk through every possible explanation — at least four distinct failure modes — and for each one, describe how you'd verify it using direct Redis inspection commands.

Problem 2 — Configuration Design

You're building a video transcoding pipeline. Each job downloads a video file (30 seconds), transcodes it (5 minutes, CPU intensive), and uploads the result (60 seconds). You have 4 servers with 8 CPU cores each. Design the complete BullMQ configuration:

• How many worker processes per server?
• What concurrency setting per Worker?
• What stalledInterval?
• Do you need job.extendLock()? How often?
• What retry policy (attempts and backoff)?
• What rate limiter settings, if any?

Justify every number from first principles — not "I guessed" but "because the job takes X minutes and the stall interval needs to be Y."

Problem 3 — The Flow Design

You're building the PROVEN hiring platform's assessment pipeline. When a candidate submits a GitHub profile: (1) fetch all their repositories in parallel, (2) for each repository, run code quality analysis in parallel, (3) after all analyses complete, generate the overall score. The overall score job must not start until all code analyses are done.

Design this as a BullMQ Flow. What are the parent and child job types? What queues do they live on? What data does the parent job receive when it runs (what does BullMQ pass to it from the children)? What happens if one code analysis fails?

Problem 4 — Scaling Under Load

Your SaaS launches on Product Hunt. In the first hour, 10,000 users sign up and each triggers a welcome email job. Your email worker normally processes 100 jobs per minute. The queue grows from 0 to 10,000 in 10 minutes. Your email provider rate-limits you at 200 emails per minute.

Describe: (a) what the Redis state looks like at minute 5, minute 30, and minute 60; (b) how to configure BullMQ's rate limiter to respect the provider limit without wasting capacity; (c) whether adding more worker processes helps in this scenario, and why; (d) what backpressure mechanism prevents the producer from adding jobs faster than they can ever be consumed.

Problem 5 — Go Deeper (connects to Chapter 2: stall detection)

BullMQ's stall detection has an inherent tension: the stalledInterval must be longer than the longest legitimate job, but shorter than the time you're willing to wait for recovery after a worker crash.

Design an alternative stall detection mechanism that doesn't have this tension — where long-running legitimate jobs are not falsely detected as stalled, and crashed workers are detected quickly regardless of how long their jobs were running. Your mechanism should work in a distributed setting with multiple workers and use only Redis primitives you've learned. Compare your approach to BullMQ's actual mechanism: in what scenarios is yours better? In what scenarios does BullMQ's approach win?

What's Coming Next

You now know Redis at the memory level, job queues at the failure-mode level, and BullMQ at the implementation level. The natural next layer is distributed systems: when your SaaS grows beyond one Redis instance and one server, what breaks and how do you fix it? That chapter covers Redis Cluster (sharding, hash slots, cross-slot limitations), Redis Sentinel (high availability and automatic failover), and the distributed systems concepts — consistency, availability, partition tolerance — that determine how you choose between them. Everything in this chapter about hash tags, atomic Lua scripts, and connection topology was preparation for that conversation.

Final Vocabulary Map — Full Curriculum Reference