Q37 — What is eventual consistency — why does it exist and what does it cost you?

What does "eventually consistent" actually mean?

Eventual consistency is a specific guarantee about what a distributed system promises: if no new writes occur, all replicas will eventually converge to the same value. "Eventually" is deliberately vague — it means at some point in the future, not at some bounded time. A system that takes 5 milliseconds to converge and one that takes 30 minutes are both eventually consistent by definition.

This is a weaker guarantee than strong consistency, which says every read returns the most recent write, always, with no staleness ever. Eventual consistency says: the data will converge, but right now, reading from different replicas might give you different answers.

Why would anyone accept such a weak guarantee? Because strong consistency has real costs that eventually consistency avoids — and for many applications, the costs of strong consistency are higher than the risks of occasional staleness.

Why does strong consistency cost so much?

To understand why eventual consistency exists, you need to understand what strong consistency requires.

Strong consistency means every read reflects the most recent committed write. In a distributed system with multiple replicas, this requires that before a write is acknowledged, all replicas agree it happened. Or at minimum, a quorum (majority) of replicas agree — so that any subsequent read, from any quorum of replicas, will include at least one node that saw the write.

This coordination takes time — at minimum, one network round trip to reach replicas. If replicas are in the same datacenter, this is 1–5ms. If replicas are in different datacenters across continents, this is 50–200ms. Every write pays this coordination cost.

Worse: if a replica is unreachable (network partition, server crash, maintenance), the system must decide: block the write until the replica comes back (maintaining strong consistency but losing availability), or allow the write to proceed and accept that the unreachable replica is temporarily inconsistent (gaining availability but relaxing to eventual consistency).

This is the fundamental trade-off the CAP theorem (Q39) describes: in the presence of a network partition, you choose between consistency and availability. Eventual consistency is the choice to keep availability.

What makes eventual consistency acceptable for many applications?

The key insight: most data in most applications is not read simultaneously by millions of people who need to all see exactly the same value at exactly the same instant.

Consider a user's post on social media. It's written by one user and read by their followers. If one follower sees the post 500ms before another follower, is this a problem? No — nobody knows the "real" time the post became visible, and sub-second differences are imperceptible.

Consider a product's view count. It's incremented on every view and displayed on the product page. If the displayed count is 1,532 when the "true" count is 1,538, does anyone notice or care? The number is approximate anyway — it's a display metric, not a transaction.

Consider DNS — the ultimate eventually consistent system (covered in Q07). When you update a domain's IP address, the change propagates across DNS servers worldwide over minutes to hours. During that window, some clients resolve to the old IP and some to the new. This is acceptable because DNS TTLs are advertised and expected.

For all of these, eventual consistency is not a defect. The staleness window (milliseconds to seconds for most in-datacenter replication) is shorter than users can perceive, and the incorrect data causes no incorrect decisions.

What does eventual consistency cost you — where it actually breaks things?

Stale reads after writes (the read-your-own-writes problem): A user updates their email address. The write goes to the primary. The user's profile page re-fetches their data — from a lagging replica — and shows the old email. The user thinks the update failed. For any user-facing write that the user immediately verifies, eventual consistency creates a confusing experience. Fix: always read back the user's own writes from the primary, route other reads to replicas.

Lost writes (last-write-wins conflicts): Two clients concurrently update the same data on different replicas. Both writes succeed locally. When the replicas sync, they have conflicting values. Some resolution is needed. The simplest strategy — last write wins, based on timestamp — can silently discard one of the two writes. If two users edit the same document simultaneously and last-write-wins applies, one user's changes are silently thrown away. For collaborative editing, this is catastrophic.

Fix: CRDTs (Conflict-free Replicated Data Types) — data structures designed so concurrent modifications can always be merged without conflicts. Counters, sets, and sequences have CRDT implementations. Complex business logic does not.

Invariant violations: Suppose you have a business rule: a user can have at most 5 active subscriptions. Two concurrent requests, both checking "does user X have fewer than 5 subscriptions?" might both read "4 subscriptions" from their respective replicas and both conclude "yes, add one more." Both writes succeed. Now the user has 6 subscriptions — a business rule violation. Eventually consistent systems cannot enforce cross-replica invariants without coordination.

Fix: for critical invariants, enforce them at write time with coordination (which means strong consistency for that specific operation), or accept the violation and handle it at the application level (detect-and-correct).

Causality violations: User A posts "meeting at 3pm." User B reads A's post and replies "I'll be there." User C reads B's reply (which their replica has) but hasn't received A's original post yet (their replica is lagging). C sees a reply without context. Effects appear before causes.

Fix: causal consistency — track causal dependencies between operations and ensure that if replica R has seen effect E, it also has seen all causes of E. This is stronger than eventual consistency but weaker than strong consistency.

What are the consistency models between eventual and strong?

It's not a binary choice. There's a spectrum:

Eventual consistency: Convergence is guaranteed, timing is not.

Monotonic read consistency: Once you've read value V, you'll never read an older value. No time-backward reads. Weaker than strong, stronger than pure eventual.

Read-your-own-writes: Your own writes are always visible to your own subsequent reads. Others might still see stale data.

Causal consistency: Causally related operations are seen in causal order. If A happened-before B, nobody sees B without also seeing A.

Sequential consistency: All clients see operations in the same order, but not necessarily in real-time order.

Linearizability (strong consistency): Every operation appears to take effect at a single instant between its start and end. The strongest model — real-time order enforced.

Many distributed systems (Cassandra, DynamoDB) let you tune the consistency level per-operation. Read quorum of ALL nodes → strong consistency. Read quorum of one node → eventual. The application chooses the trade-off for each operation based on what it needs.

The mental model

Strong consistency: every read returns the most recent write — always
  Cost: coordination on every write (network round trips), blocks on partition

Eventual consistency: replicas will converge, eventually
  Benefit: low write latency (no cross-replica coordination), available during partition
  Cost: stale reads, conflict resolution needed, invariants can be violated

Where eventual consistency is fine:
  Social media feeds, view counts, DNS, product catalogs, analytics
  (Staleness is imperceptible or harmless)

Where eventual consistency breaks things:
  Read-your-own-writes: fix with primary reads for own-user data
  Last-write-wins conflicts: fix with CRDTs or application-level merge
  Invariant enforcement (max 5 subscriptions): fix with coordination or detect-and-correct
  Causality violations: fix with causal consistency tracking

Spectrum (weakest → strongest):
  Eventual → Monotonic read → Read-your-own-writes → Causal → Sequential → Linearizable

Tunable consistency (Cassandra/Dynamo):
  Per-operation choice: quorum size determines the consistency level you get
  High quorum → stronger consistency, higher latency
  Low quorum → weaker consistency, lower latency

Eventual consistency exists because strong consistency is not free — it costs latency, availability, and throughput. The question is never "strong vs eventual" in the abstract. It's "for this specific piece of data, what level of staleness is acceptable, and what are the consequences of getting it wrong?" Different answers for different data types lead to different consistency choices, sometimes within the same application.