Q30 — What is a circuit breaker and why does it exist?
What failure does a circuit breaker prevent?
In a distributed system, services call other services. If Service B is down or severely degraded, what should Service A do when it tries to call B?
The naive answer: try the call, wait for the timeout, get an error, maybe retry, return an error to the caller.
This seems reasonable for one request. But at scale, it's catastrophic.
Suppose Service A handles 10,000 requests per second, each of which calls Service B. Service B goes down. Now:
- Every one of those 10,000 req/s tries to call B and waits for the timeout (say, 5 seconds)
- Service A has 5 seconds × 10,000 req/s = 50,000 concurrent in-flight requests to B, all blocked waiting for timeouts
- Each blocked request holds a thread, a connection, and memory in Service A
- Service A's thread pool exhausts. New incoming requests to A can't be served
- Service A is now down — not because A had a problem, but because B was down
Service A's failure caused by B's failure is called cascading failure. The failure propagates upstream through the call graph. A single failing service takes down everything that depends on it — which takes down everything that depends on those services.
The underlying mechanism is resource exhaustion under slow failures. A fast failure (connection refused) is fine — the request fails quickly and resources are freed. A slow failure (timeout) is dangerous — resources are held for the entire timeout duration, and 10,000 slow failures simultaneously exhaust capacity.
The circuit breaker pattern prevents this.
Where does the name come from?
An electrical circuit breaker is a safety device. When current exceeds a safe threshold (a short circuit or overload), the circuit breaker trips — it opens the circuit and stops current from flowing. This prevents the wiring from overheating and causing a fire. Once the problem is fixed, the breaker is reset and current flows again.
The software circuit breaker does the same thing logically: when failures exceed a threshold, it "trips" — stops forwarding requests to the failing service. This prevents the cascading resource exhaustion. Once the service recovers, the circuit resets and normal operation resumes.
What are the three states of a circuit breaker?
CLOSED — normal operation
The circuit is closed (current flows). Requests to the downstream service proceed normally. The circuit breaker tracks the failure rate of recent requests (typically over a rolling time window or a fixed sample count).
If the failure rate stays below the threshold, nothing changes. If the failure rate exceeds the threshold (say, 50% of the last 100 calls failed), the circuit breaker trips: it transitions to OPEN.
OPEN — failing fast
The circuit is open (current doesn't flow). No requests are forwarded to the downstream service at all. Instead, every request is immediately failed — without trying the call — with a specific error (like a CircuitOpenException).
Why fail immediately rather than try? Because the downstream is known to be failing. Trying the call just adds timeout latency and resource consumption for a call that will fail anyway. Failing fast means requests fail in microseconds instead of seconds, freeing resources immediately. This protects Service A from the resource exhaustion that causes cascading failure.
The circuit stays OPEN for a configured duration (the "sleep window" — typically 10–60 seconds).
HALF-OPEN — testing recovery
After the sleep window expires, the circuit transitions to HALF-OPEN. It allows a small number of probe requests through to the downstream service — typically just one at a time.
If the probe succeeds, the downstream service has recovered. The circuit transitions back to CLOSED. Normal traffic resumes.
If the probe fails, the downstream is still unhealthy. The circuit transitions back to OPEN for another sleep window. This prevents flooding a recovering service with full traffic before it's actually ready.
What does the circuit breaker give the downstream service?
When the circuit is OPEN, zero requests reach the downstream service. This is intentional and valuable. A service that's failing is often failing because it's overwhelmed — database connections saturated, thread pool exhausted, memory pressure causing slow GC. Continuing to send it requests makes recovery harder.
With the circuit open, the downstream gets a quiet period: no new request pressure, time for connection pools to drain, threads to free, databases to catch up. This breathing room is often the difference between a 30-second blip and a 30-minute outage.
The circuit breaker is a form of automatic load shedding — protecting a struggling downstream by stopping the inbound flow before the downstream tells you to.
How does the circuit breaker interact with retries?
Circuit breakers and retries work together but at different levels.
Retries handle transient single-request failures — the occasional dropped packet, the momentary resource spike. A request fails, waits a moment, retries, succeeds. Fine.
Circuit breakers handle sustained downstream degradation — when failures are persistent and systemic. If the circuit is open and you're retrying into it, you're not retrying into the actual service (the circuit fails fast), so your retry amplification is bounded. The circuit breaker prevents retries from reaching a struggling service.
The coordination: when a circuit is open, your retry logic should not retry calls that fail due to the open circuit — those failures are intentional, not transient. Only retry when the circuit is CLOSED and you receive a genuine service error.
What does the caller do when the circuit is open?
The circuit breaker fails fast, but "failing fast" to the user is still a failure. The system should have a plan for what to show or do when the downstream is unavailable. Common strategies:
Fallback response: Return a degraded but useful response. If the recommendation service circuit is open, return the user's most recently successful recommendations (cached) instead of nothing. If the inventory service is open, show "check availability" instead of the exact count.
Fail silently: If the downstream is non-critical (telemetry, logging, analytics), drop the call silently and move on. The user's request succeeds; the non-critical data is just missing.
Fail visibly: If the downstream is critical and no fallback exists, return an error to the user. This is correct — the user should know their action didn't complete, rather than receiving false confirmation.
The circuit breaker tells you that you need a fallback strategy. What that strategy is depends on what the service does and what partial degradation is acceptable.
The mental model
Without circuit breaker:
B fails → A waits timeout per request → N concurrent requests × timeout duration
→ A's threads/connections exhausted → A fails → cascading upstream
Circuit breaker states:
CLOSED (normal):
→ Requests flow through to B
→ Track failure rate in rolling window
→ Failure rate > threshold → trip → OPEN
OPEN (protecting):
→ All requests fail immediately (microseconds, not seconds)
→ A's resources freed instantly
→ B gets quiet recovery time
→ After sleep window → HALF-OPEN
HALF-OPEN (testing):
→ One probe request to B
→ Probe succeeds → CLOSED (B recovered)
→ Probe fails → OPEN again (B still struggling)
Interaction with retries:
Retries = handle transient single-request failures
Circuit breaker = handle sustained downstream degradation
Circuit open → don't retry (failures are intentional fast-fails)
When circuit is open, have a plan:
Cached fallback / degraded response / silent drop / explicit error
Real implementations: Hystrix (Netflix, now maintenance), Resilience4j (Java),
Polly (.NET), go-breaker (Go)
The circuit breaker exists because a slow failure is more dangerous than a fast one. It converts slow, resource-exhausting timeouts into fast, resource-sparing rejections — protecting the caller from being dragged down by a struggling dependency, and protecting the dependency by stopping the inbound load during recovery.