Chapter 6: Consistency vs Availability — What "Consistent" Actually Means

Curriculum: Distributed Systems from First Principles Chapter: 6 of 24 Part: 2 — Replication Prerequisites: Chapter 4 (replication motivation, sync vs async), Chapter 5 (WAL replication mechanics, replication lag, split-brain) Estimated reading time: 80–90 minutes

Opening

"We use eventual consistency" and "our system is strongly consistent" are phrases you'll hear constantly in engineering discussions, interviews, and architecture reviews. Most engineers who say them don't know what they mean precisely enough to make architecture decisions from them. "Consistent" is one of the most overloaded words in distributed systems — it means four different things depending on who's speaking, and conflating them leads directly to production bugs, over-engineered systems, and interview answers that sound knowledgeable but are technically wrong. This chapter corrects that. You'll learn that consistency is not a binary property but a spectrum with four distinct levels, each with a precise physical definition and a measurable cost. You'll see the CAP theorem as it was actually stated — not the popular "pick two" myth — and understand what it actually constrains. You'll learn why CAP is almost never the relevant trade-off in day-to-day engineering, and meet PACELC, which is. By the end, you'll be able to look at any piece of data in your SaaS product and answer the question: what is the minimum consistency level this data requires, and what is the cost of providing it?

Part 1: The Word "Consistent" Is Doing Too Much Work

Before anything else, you need to understand that the word "consistent" means different things in different contexts, and conflating them causes enormous confusion.

Consistent in ACID (what you learned first, from SQL databases): after a transaction commits, the database is in a valid state — all constraints are satisfied, foreign keys are intact, no invariants are violated. This is about data integrity within one database, not about multiple machines seeing the same value.

Consistent in CAP theorem: every read receives the most recent write or an error. This is specifically about whether clients see the same view of data across multiple nodes. It's a property of a distributed system, not a single database.

Consistent in "consistent hashing" (Chapter 8): a specific algorithm for distributing keys across nodes. Nothing to do with the other two.

Consistent in everyday speech: "our data is consistent" usually means "not corrupted" or "not contradictory" — a vague colloquial use.

Throughout this chapter, "consistency" means the distributed systems definition: what guarantees does the system make about what a client observes when reading data from any node?

This definition immediately reveals a spectrum. A system can give strong guarantees ("you always read the latest value") or weak ones ("you'll eventually read the latest value, sometime"). Between those extremes are two more positions. The four levels, from strongest to weakest, are: linearizability, sequential consistency, causal consistency, and eventual consistency. Each represents a different physical cost and a different set of observable anomalies.

Part 2: The Consistency Spectrum

Level 1: Linearizability — The Gold Standard

Linearizability is the strongest consistency guarantee a distributed system can offer. Its formal definition is precise: every operation appears to take effect instantaneously at some single point in time between its invocation and its completion, and once a write completes, all subsequent reads from any client on any node see the written value.

That's the definition. Here's what it means physically.

Imagine a single machine with a single piece of memory holding the value x = 0. Client A writes x = 1. The moment that write completes — the moment the machine acknowledges it — any client that reads x, from anywhere, sees 1. There's no window during which some clients see 0 and others see 1. The write appears to happen at a single instant, visible to everyone simultaneously.

Linearizability demands the same behaviour across a distributed system. After a write completes on any node, every subsequent read from every other node must see that write. There is no staleness, no "the replica hasn't received it yet" — from the client's perspective, the system behaves as if it's a single, infinitely-fast, globally-shared piece of memory.

The physical cost of linearizability in a replicated system: to guarantee that a read returns the latest write, you either (a) route all reads to the primary (the only node guaranteed to have the latest value), or (b) use synchronous replication with remote_apply — the primary doesn't acknowledge a write until all replicas have applied it, so any node can serve reads. Both options add latency. Option (a) means replicas can't serve reads that need freshness. Option (b) means every write waits for the slowest replica.

In practice, linearizability is expensive enough that it's used sparingly — only for data where any stale read causes a correctness problem. Financial balances. Distributed locks. Configuration that controls system behaviour. Inventory counts where overselling is unacceptable.

Level 2: Sequential Consistency

Sequential consistency relaxes one constraint from linearizability: operations must appear in some sequential order that is consistent with each individual client's order, but there's no requirement that this order corresponds to real wall-clock time.

The difference sounds subtle but matters. With linearizability: client A writes x = 1 at time T; client B reads x at time T+1ms and must see 1. The real-time ordering is respected.

With sequential consistency: client A writes x = 1. Client B reads x. Client C reads x. Sequential consistency only guarantees that B and C see the same sequence of operations — they might both see x = 0 if the write hasn't propagated yet, as long as they both see the same thing and the write eventually appears in the global order. The real-time constraint — "it has been 1ms since the write" — is not enforced.

This is a weaker guarantee, but it eliminates most real-world anomalies. It's the consistency level offered by many multi-processor CPU architectures (with memory barriers) and some distributed databases. In practice, the gap between linearizability and sequential consistency is often small, and sequential consistency is rarely the right choice — systems either go stronger (linearizability) or weaker (causal or eventual). But understanding it fills in the spectrum.

Level 3: Causal Consistency

Causal consistency is where distributed databases get practically interesting. Instead of requiring global agreement on all operation orders, it only requires agreement on operations that are causally related.

Two operations are causally related if one could have influenced the other:

• A write and a subsequent read of the same value (the reader might have seen the written value)
• A read and a subsequent write (the writer might have based their write on the read value)
• Any two operations where the second was performed after observing the result of the first

Causally unrelated operations (concurrent writes to different keys by clients who have not communicated) can be seen in any order by different nodes.

The classic example is a social network. Alice posts: "I'm getting married!" Bob reads Alice's post and replies: "Congratulations!" Now Carol loads the thread. Under causal consistency, Carol is guaranteed to see Alice's original post before Bob's reply — because Bob's reply is causally dependent on Alice's post (he couldn't have written the reply without seeing the post). But if Dave simultaneously posts "Happy Monday!" with no causal relationship to Alice's post, Carol might see Dave's post before Alice's. That's acceptable under causal consistency.

Causal consistency is significantly cheaper to implement than linearizability, because causally unrelated operations don't need to be globally coordinated. Nodes only need to track causal dependencies (usually via vector clocks or causal tokens), not global wall-clock order. This makes it viable for geo-distributed systems where the latency of global coordination would be prohibitive.

MongoDB's causally consistent sessions, and some configurations of DynamoDB with conditional writes, approximate causal consistency. It's the strongest consistency level that remains practically achievable across distant datacenters without enormous latency overhead.

Level 4: Eventual Consistency — The Weakest Guarantee

Eventual consistency makes one promise: if no new updates are made to a data item, eventually all replicas will converge to the same value. That's it.

It says nothing about:

• How long "eventually" takes (milliseconds? hours? it depends on the system)
• What value the replicas will converge to (the latest write? some merge? it depends on the conflict resolution policy)
• What a client reads in the meantime (anything is permitted — stale values, old values, even values that were never written if the conflict resolution is poorly designed)

Werner Vogels (CTO of Amazon) described eventual consistency in 2007 as a spectrum of weaker guarantees that each add specific properties:

• Monotonic read consistency: if you've read a value, you'll never read an older one. (Sequential reads don't go backwards in time.)
• Read-your-own-writes: after you write a value, your subsequent reads see it. (Your own writes are not invisible to you.)
• Monotonic write consistency: writes from the same client are applied in order. (Your writes are not reordered.)
• Write-follows-reads: if you read a value and then write based on it, the write is ordered after the read. (The write causally follows the read.)

Each of these is a consistency guarantee stronger than pure eventual consistency but weaker than linearizability. Systems like DynamoDB, Cassandra (with eventual consistency setting), and Riak offer eventual consistency as a baseline and let you opt into stronger properties per-operation.

The critical misunderstanding about eventual consistency: it does not mean your data will be eventually correct. It means replicas will converge. If the conflict resolution is "last write wins by timestamp" and two clients write simultaneously with slightly different clocks, one write will be silently discarded. That's eventual consistency delivering convergence — the replicas agree on the value — but the application may consider the discarded write a data loss event. Eventual consistency guarantees convergence, not correctness. The application must be designed to tolerate or prevent the specific anomalies that eventual consistency permits.

Part 3: CAP Theorem — What It Actually Says

In 2000, Eric Brewer gave a keynote at PODC (Principles of Distributed Computing) proposing a conjecture: a distributed system can have at most two of three properties — Consistency, Availability, and Partition Tolerance. This became known as the CAP theorem. Gilbert and Lynch formally proved it in 2002.

Here is the problem with every popular explanation you've read: they tell you "pick two of three, and since you can't avoid partitions, you really just pick between C and A." This framing is wrong in two important ways.

Wrong framing #1: "Consistency" in CAP means linearizability specifically.

Not "your data is consistent" in any colloquial sense. Not ACID consistency. Not "replicas eventually agree." CAP's C means linearizability — the strictest definition from Part 2. A system that is "eventually consistent" already sacrifices CAP's C. A system with ACID transactions on a single node is consistent in the ACID sense but may not be consistent in the CAP sense if it has replicas that can serve stale reads.

Wrong framing #2: "Partition tolerance" is not a choice.

Network partitions — failures where some nodes can communicate with each other but not with others — are a physical reality. Your datacenter's switch will fail. The network cable will be cut. A cloud provider will have a routing issue. If you build a system that "doesn't tolerate partitions," what you've actually built is a system that becomes completely unavailable when a partition occurs. That's not "not tolerating partitions" — it's choosing unavailability as your partition response. Partition tolerance isn't a feature you can omit; it's a description of whether your system responds to partitions by staying up (at some cost) or going down.

The correct framing:

CAP theorem says: during a network partition, you cannot guarantee both that (a) every read returns the most recent write (linearizability) and (b) every request receives a response (availability). You must choose which to sacrifice.

This is a narrower, more precise claim. It doesn't say you can't have both C and A in normal operations. It says you can't have both simultaneously when a partition is actively occurring.

Let's trace exactly why.

Before the proof, cut the link yourself. Predict what the cut-off side returns under each choice — the next section explains why there is no third option.

Part 4: The Partition Scenario — Why C and A Are Incompatible

You have two nodes: Node A (primary) and Node B (replica). A network partition occurs. Node A can't communicate with Node B. Clients can reach both nodes. A client writes balance = 100 to Node A. The write succeeds on Node A. Node A cannot ship this write to Node B.

Now a different client reads balance from Node B.

If you choose Consistency (linearizability): Node B must refuse to serve the read. It knows it might be out of sync — it cannot confirm it has the latest value without contacting Node A, which it cannot reach. To maintain linearizability, it must return an error: "I cannot guarantee I have the latest data. Try again later or contact another node." The client gets no response (or an error). Availability is sacrificed.

If you choose Availability: Node B serves its current value: balance = 0 (it hasn't received the write). The client reads stale data. A third client reads from Node A and sees balance = 100. Two clients, reading simultaneously from the same distributed system, see different values. This violates linearizability. Consistency is sacrificed.

You cannot escape this. There is no third option. Given a partition, a node must either refuse to answer (sacrificing availability) or answer with potentially stale data (sacrificing linearizability). The laws of physics — specifically the finite speed of light and the fact that network messages can be lost — make this inescapable.

In practice, the choice has a time dimension: how long does the partition last? If a network switch fails and is replaced in 30 seconds, a CP system is unavailable for 30 seconds. An AP system serves stale data for 30 seconds. Which is worse depends entirely on the application. A bank account balance: CP (stale balance might cause overdrafts). A social media like count: AP (showing 1,247 likes instead of 1,248 for 30 seconds is fine).

Most real systems provide tunable consistency — you can choose per-operation whether to require quorum (CP-leaning) or accept any available replica (AP-leaning). Cassandra's consistency levels (ONE, QUORUM, ALL) are exactly this dial. DynamoDB's ConsistentRead flag is exactly this dial.

Part 5: PACELC — The Trade-off CAP Ignores

CAP theorem only describes what happens during a partition. Partitions are rare. What about the other 99.9% of the time, when the network is healthy?

In 2012, Daniel Abadi proposed the PACELC framework, which extends CAP to cover the normal case:

• P: if there is a Partition → choose Availability or Consistency (this is CAP)
• EL: Else (no partition) → choose Latency or Consistency

The "EL" part is where most real engineering decisions live. Even when the network is healthy and there are no partitions, there is an unavoidable trade-off between how fast you can serve a read and how consistent that read is.

Why latency and consistency trade off in normal operation:

To serve a linearizable read, you must ensure you're reading the most recent value. In a replicated system, that means either:

1. Always reading from the primary (which may be geographically distant, adding latency).
2. Doing a quorum read — confirming with a majority of replicas that you have the latest value before returning (adds round-trip latency).

To serve a low-latency read, you read from the nearest replica and return immediately — no coordination needed. That replica might be slightly behind, so you sacrifice consistency.

Every query to a distributed database makes an implicit choice on this latency-consistency axis. The PACELC framework makes it explicit.

Placing real systems on the PACELC spectrum:

SYSTEM               PARTITION BEHAVIOUR    NORMAL BEHAVIOUR
--------             -------------------    ----------------
Postgres (sync)      PC (refuse writes       EC (low latency sacrificed
                     without quorum)         for strong consistency)

DynamoDB (eventual)  PA (serve stale reads)  EL (fast reads, stale possible)

DynamoDB (strong)    PC (quorum read)        EC (slower read for freshness)

Cassandra (ONE)      PA                      EL

Cassandra (QUORUM)   PC                      EC

Spanner              PC                      EC (TrueTime for consistency)

Redis (no replica)   N/A (single node)       EL (in-memory = fast)

Redis (async replica)PA                      EL

Notice that Cassandra and DynamoDB appear in both categories — they're tunable. You choose per-operation. This is the right design for a system serving multiple types of data with different consistency requirements.

The takeaway from PACELC: in normal operation, you're always trading latency for consistency. This trade-off exists even when your cluster is healthy and no partitions are occurring. Every time you read from a replica instead of the primary, you're making the EL trade-off: lower latency, weaker consistency. Every time you require a quorum read, you're making the EC trade-off: stronger consistency, higher latency.

Part 6: Write Conflicts — When Multiple Nodes Accept the Same Write

The chapters so far have dealt with single-leader replication — one primary accepts writes, standbys receive them. In this model, write conflicts don't exist. The primary serialises all writes. If two clients write the same key simultaneously, the primary processes them in some order and the later write wins. There's no ambiguity.

But some systems accept writes on multiple nodes simultaneously — multi-leader replication and leaderless replication (Dynamo-style, Chapter 8). When two clients write the same key on different nodes at the same time, a conflict arises. Both writes are "valid" from the respective node's perspective. When the nodes synchronise, they discover they have different values for the same key. What do they do?

This is not a problem you can architect away. In any system that accepts writes on multiple nodes without coordination, conflicts are possible. The question is how you resolve them.

Strategy 1: Last Write Wins (LWW)

Each write is tagged with a timestamp. When two conflicting writes are discovered, the one with the later timestamp wins, and the other is discarded.

The problem: clocks are not perfectly synchronised across machines. If Node A's clock is 50 ms ahead of Node B's clock, a write at "time 1000" on Node B loses to a write at "time 1049" on Node A — even if the Node B write happened after the Node A write in real time. Node A's earlier write (in real time) wins because its clock is ahead.

This is clock skew. NTP synchronises clocks to within tens of milliseconds, but "tens of milliseconds" is enough to silently discard real, legitimate writes. Systems like Cassandra use LWW by default, which means they can silently drop writes under concurrent modification. For a shopping cart or a financial balance, this is a correctness problem. For a "last seen online" timestamp or a social media like count, it might be acceptable.

LWW is simple to implement and fast, which is why it's common. Its danger is that data loss is silent — the system doesn't return an error; the write just disappears.

Strategy 2: Version Vectors / Vector Clocks

Instead of a single timestamp, track a vector of counters — one per node. Each node increments its own counter on every write. When nodes synchronise, they compare version vectors to determine if one value is "newer" than another or if the values are "concurrent" (neither supersedes the other).

Node A writes x=1:  version = {A:1, B:0}
Node B writes x=2:  version = {A:0, B:1}

Are these concurrent? Neither version vector is ≥ the other.
Node A's version has A:1 > A:0, but B:0 < B:1.
Node B's version has B:1 > B:0, but A:0 < A:1.
Conclusion: concurrent writes. CONFLICT.

When a conflict is detected via version vectors, the system surfaces both values to the application. The application resolves it: take the higher value, merge them, ask the user, or apply domain-specific logic.

Amazon's Dynamo (the paper, not DynamoDB the product) uses vector clocks for this. When you read from Dynamo with a conflict, you get multiple values back and must write a merged version. In the original Dynamo, this was called "reconciliation on read" — conflicts are deferred until the next read.

The downside of vector clocks: they grow as the number of nodes grows. In a system with 100 nodes, every version vector is a 100-element array. Truncation strategies (pruning old entries) can re-introduce the inability to detect some conflicts. And requiring the application to resolve conflicts adds complexity — the application must understand the domain semantics of the data to merge it correctly.

Strategy 3: CRDTs (Conflict-free Replicated Data Types)

A CRDT is a data structure mathematically designed so that any two states can always be merged into a consistent result, regardless of the order operations were applied. Concurrent modifications never create conflicts — they always have a deterministic merged result.

The classic example is a counter that can only be incremented. If Node A increments to 5 and Node B increments to 3 (from an initial 0), the merge is simple: take the maximum of each node's count (since each node tracks its own increments separately), and sum them. The result is always deterministic: 5 + 3 = 8.

More complex CRDTs exist for sets (G-Set, 2P-Set), maps, text (for collaborative editing, like the operational transforms used in Google Docs), and registers.

The limitation: not all data is CRDT-compatible. A bank account that allows both deposits and withdrawals has the possibility of going negative from concurrent withdrawals. The "never-negative" constraint cannot be maintained without coordination. You cannot CRDT your way out of a consistency requirement that is fundamentally about preventing an invalid state.

Understanding these three strategies matters for system design interviews and architecture decisions. When you choose a database that uses eventual consistency, the next question is always: what is its conflict resolution strategy? If the answer is LWW, is your data safe under concurrent writes? If the answer is "application resolves conflicts," are you prepared to write that resolution logic?

Part 7: Consistency Requirements by Data Type

Now you have the vocabulary to make precise statements about consistency requirements. For each type of data in your SaaS product, you can state: what is the minimum consistency level required, what does providing it cost, and what anomalies am I willing to accept?

Financial balances, account balances, anything money-related: Requires linearizability. Two concurrent reads of the same balance must see the same value. A debit and a credit applied simultaneously must not result in a negative balance if the account doesn't have the funds. LWW is dangerous — a concurrent deposit and withdrawal might silently lose one operation. This data belongs on a single-leader system with synchronous replication, or a system with strong consistency guarantees (Spanner, CockroachDB).

Subscription status, feature flags, entitlements: Requires at minimum read-your-own-writes. After a user upgrades, they must see Pro features immediately. Other users seeing your old tier briefly is more acceptable. Route post-write reads to the primary for the writing user. Read replicas are fine for other users' data.

User-generated content (posts, comments, messages): Causal consistency is usually sufficient and sometimes required. Bob's reply must appear after Alice's post (causal dependency). Two unrelated posts by different users can appear in any order to different observers. This is achievable with causal consistency and dramatically cheaper than linearizability at geographic scale.

Aggregate counts (likes, views, follower counts): Eventual consistency is almost always fine. The exact count being 1,247 vs 1,248 is rarely product-critical. Systems like Redis counters with async sync to DB, or approximate counting structures like HyperLogLog, are appropriate here.

Distributed locks (e.g., "only one worker processes this job"): Requires linearizability. A lock is only useful if acquiring it is globally visible — if two nodes can both believe they hold the lock simultaneously, the lock provides no protection. Redis's Redlock algorithm attempts distributed locking over multiple Redis instances; its consistency guarantees are debated. For hard correctness requirements, use a consensus-based lock service (etcd, ZooKeeper).

Search indexes, caches, derived data: Eventual consistency is usually fine. These are derived from the authoritative source. If they're stale by seconds or minutes, that's acceptable because they're not the source of truth — they can be invalidated and rebuilt.

Audit logs: Requires at minimum monotonic write consistency — entries must be appended in the order they happened. Linearizability is ideal (so audit logs can be used as evidence of the exact sequence of events), but causal consistency is often sufficient if timestamps are attached to each event.

The exercise to do for your own SaaS product: list every table in your database. For each one, answer: what consistency level does it actually need? How often is it written to concurrently by multiple clients? What anomaly would occur if a read returned a stale value? You'll likely find that 80% of your data works fine with eventual consistency (or read-your-own-writes), and 20% needs strong consistency. The 20% is where you focus your engineering effort and where you justify the latency cost.

Part 8: The Decision Framework

When choosing a consistency level for a piece of data or a system component, answer these four questions in order:

Question 1: What is the worst-case consequence of a stale read?

Inconvenience (user sees old profile picture): eventual consistency fine. Product defect (user sees old plan status): needs read-your-own-writes minimum. Correctness error (balance shows more money than exists): needs linearizability. Safety issue (configuration controls behaviour of a critical process): needs linearizability.

If the answer is "nothing bad happens, the value is corrected on the next read," eventual consistency is appropriate.

Question 2: Are concurrent writes possible, and what happens if two conflict?

If only one client writes this data at a time, conflicts don't occur. Single-leader replication with async standbys is sufficient.

If multiple clients write concurrently, ask: is LWW acceptable (one write is silently discarded)? Is the data CRDT-compatible (counts, sets, max values)? Does the application need to resolve conflicts explicitly?

Question 3: What is the latency budget for reads?

If reads must be under 10 ms and the primary is 150 ms away, you cannot linearise reads through the primary. You must use a local replica and accept weaker consistency. This is PACELC's EL trade-off materialising as a real constraint.

Question 4: What is the write throughput requirement?

Linearisable writes (synchronous replication, quorum writes) add latency and reduce throughput. If your system needs 100,000 writes/second, requiring quorum confirmation for each write means 100,000 × (network round trip × number of nodes) of coordination overhead per second. At some write rate, the coordination cost overwhelms the system. High-throughput write systems almost always use eventual or causal consistency.

Vocabulary Map

Practice Problems

Problem 1 — Failure Analysis

Your SaaS product has a multi-region deployment: primary in US-East, read replica in EU-West (async, typically ~80 ms lag). You use eventual consistency for most reads and route to the nearest region.

A user in Europe sets their account password at 14:00:00. The write goes to US-East primary (because all writes go to primary). At 14:00:01 (one second later), the same user attempts to log in. The login request hits the EU-West replica. The password change has not yet replicated — the replica still has the old password hash.

What does the user experience? Classify this failure in terms of the consistency spectrum. What is the minimum consistency level needed to prevent it? Design two solutions: one that provides the minimum necessary consistency without changing your replication architecture, and one that provides stronger consistency by changing where login validation happens.

Problem 2 — Design from Scratch

You're designing the data storage strategy for a real-time collaborative document editor (like Google Docs). Multiple users can simultaneously edit the same document. The requirements are:

• Users in the same editing session must see each other's edits within 500 ms.
• Users in different sessions (or users who open the document hours later) must see a consistent final state.
• The system must work across three geographic regions.
• A user's own edits must always appear immediately in their own view.

Map these requirements to consistency levels from this chapter. For each requirement, state: which consistency level satisfies it, and which does not. Then: which of the three conflict resolution strategies (LWW, vector clocks, CRDTs) is most appropriate for document text editing, and why? What anomaly would occur if you used LWW for character insertions?

Problem 3 — Synthesis

Connect the CAP theorem, PACELC, and the four consistency levels into one answer to this interview question: "Explain how you would choose between DynamoDB and Postgres for storing user financial transaction records."

Your answer must: (a) name the specific consistency level required and explain why weaker levels are insufficient; (b) identify where on the PACELC spectrum each database sits and what that means for this use case; (c) acknowledge that the partition question (CAP's C vs A choice) is separate from the latency question (PACELC's EL choice), and address both; (d) arrive at a concrete recommendation with reasoning that references the physical constraints of replication.

Problem 4 — Go Deeper

Google Spanner is described as a "globally distributed, externally consistent database." External consistency is a stronger guarantee than linearizability — not just that reads after a write see the write, but that the system's behaviour is consistent with real-time causality even across datacenters thousands of kilometres apart.

To achieve this, Spanner uses TrueTime — an API that returns a time interval [earliest, latest] rather than a point timestamp. The interval is guaranteed to contain the true current time, and its width (typically 1–7 ms) reflects the uncertainty in clock synchronisation.

Spanner uses TrueTime to assign timestamps to transactions: a transaction's commit timestamp must be after latest (so the true commit time is within the interval after the reported time). Reads at a given timestamp T only proceed after waiting until latest > T (ensuring no future transaction with a lower timestamp can appear).

From these clues: why does uncertainty in clock time create consistency problems for distributed transactions? How does committing after latest solve the problem of a transaction appearing "in the past"? What is the trade-off — what does Spanner sacrifice to achieve external consistency via TrueTime, and why can't a system with larger clock uncertainty (say, 100 ms instead of 7 ms) afford the same approach?

Problem 5 — Connect to Previous Chapters

Chapter 5 introduced synchronous_commit = remote_apply — the primary waits for the standby to apply WAL to its heap before acknowledging the write to the client.

Using the consistency vocabulary from this chapter: what consistency level does remote_apply synchronous replication achieve? Is it linearizability? Be precise — trace through the two-node scenario (primary and one sync standby) and check whether the linearizability guarantee holds: "after a write completes, all subsequent reads from any node return the written value." Are there any edge cases where it does not hold?

Then: Chapter 4 introduced replication lag and the read-your-own-writes problem. Using this chapter's vocabulary, what consistency level does async replication with read-your-own-writes routing (write to primary, read from primary for own data, read from replica for others' data) actually provide? Name the specific guarantee and identify which guarantees it fails to provide.

What's Coming Next

Part 2 ends here. You now have the complete replication toolkit: the physical mechanism (Chapter 5), the failure model (Chapter 4), and the consistency framework that lets you reason about what any replicated system actually guarantees (this chapter). Every database, every cache, every distributed system you encounter for the rest of your career sits somewhere on the PACELC spectrum, and you can now place it there.

Part 3 begins with Chapter 7: Sharding. Replication solves fault tolerance and read scale. It does not solve write scale — all writes still go to one primary, and one machine has a finite write throughput limit. When you hit that limit, you split the data across machines. Chapter 7 is about how to split it, what it costs, and why the seemingly obvious approaches break in non-obvious ways.

Chapter 6 complete. Problem 3 is the most valuable to write out fully — it's a complete system design interview answer using everything from chapters 4, 5, and 6. Problem 4 is the hardest intellectually: TrueTime is a beautiful piece of engineering and reasoning through it from the clues alone will sharpen how you think about clock skew. Problem 5 closes the loop on chapters 4–6 — do it last.