Q48 — What is service discovery — why can't services just hardcode each other's addresses?

Why hardcoding addresses fails immediately

In a microservices architecture, services communicate over the network. The simplest mental model: Service A knows Service B is at 192.168.1.42:8080, hardcodes that address, and calls it.

This works exactly until something changes — which is always. Services crash and restart on different IP addresses. You deploy a second instance of B for load balancing; now A needs to know about both. You roll out a new version of B; during the transition, old and new instances coexist. You move B to a different host because the old one needed maintenance. You autoscale B from 2 instances to 20 because of a traffic spike.

In every one of these scenarios, the hardcoded address is wrong. A makes calls to an address that is either dead, wrong, or only a subset of available instances. The system breaks in production the moment anything changes — which is constantly.

Service discovery is the solution: rather than hardcoding addresses, services register themselves into a central registry as they start up, and clients query the registry to find available instances at call time.

What does a service registry do?

A service registry is a database of running service instances and their network addresses. When a service starts, it registers itself: "I am order-service, I am reachable at 10.0.1.15:9090." When it shuts down (gracefully), it deregisters. The registry is the source of truth for what is alive and where.

Two additional mechanisms keep the registry accurate:

Health checks: The registry periodically pings each registered instance (or instances push heartbeats to the registry). If an instance stops responding — because it crashed, hung, or got OOM-killed — the registry marks it as unhealthy and stops returning it to clients. Without this, clients would discover dead instances and fail.

TTLs and lease renewal: Some registries issue a lease: "your registration is valid for 30 seconds; renew it or you'll be removed." A healthy service renews continuously. A dead service stops renewing and is automatically evicted after the TTL expires. This handles cases where the registry itself can't actively ping the service (firewalls, one-way network policies).

Common implementations: Consul, etcd, ZooKeeper, Eureka (Netflix). In Kubernetes, the registry is etcd and the discovery mechanism is built into DNS and Services — every Kubernetes Service gets a stable DNS name that resolves to the current healthy pod IPs.

What are the two discovery patterns?

Once you have a registry, there are two places to put the discovery logic: in the client itself, or in a separate infrastructure component.

Client-side discovery: The client service queries the registry directly, gets the list of healthy instances of the target service, picks one (via its own load balancing logic — round-robin, least-connections, consistent hashing), and makes the call. The client owns the full flow: discover, load-balance, call.

Pros: simple infrastructure (just the registry), client has full control over load balancing strategy. Cons: every service must implement discovery and load balancing logic. If you have ten services in ten different languages, you implement this ten times. Any bug in the client-side load balancer is replicated across every service.

Server-side discovery (load balancer / service mesh): The client sends its request to a fixed address — a load balancer or a sidecar proxy (like Envoy in a service mesh). The proxy queries the registry, picks a healthy instance, and forwards the request. From the client's perspective, service B is always at the same address — the proxy handles the dynamic routing transparently.

Pros: discovery and load balancing logic lives in one place; clients are simple and language-agnostic. Cons: the proxy is an extra network hop; it's another piece of infrastructure to operate.

Modern systems use server-side discovery via a service mesh (Istio, Linkerd, Consul Connect): a sidecar proxy is injected alongside every service container. All inter-service traffic flows through sidecars. Discovery, load balancing, retries, circuit breaking, mTLS, and observability are handled at the sidecar layer — completely invisible to the application code.

How does DNS fit into service discovery?

DNS is the oldest and most widely understood form of service discovery. In Kubernetes, when you create a Service, the cluster DNS automatically creates a record: order-service.default.svc.cluster.local resolves to the stable virtual IP of the service. That virtual IP load-balances across healthy pods via kube-proxy.

For the calling service, discovery is as simple as using a hostname instead of an IP. The hostname is static; the resolution is dynamic. This is why Kubernetes made DNS its primary service discovery mechanism — it requires zero client changes, works with any language or library, and integrates with the existing networking model.

The limitation of pure DNS: DNS TTLs mean stale entries can persist for seconds to minutes after an instance goes down. DNS doesn't carry health status — it returns addresses, not health. For high-churn environments or strict health requirements, a registry with active health checks is more responsive.

What is the connection between service discovery and load balancing?

Service discovery and load balancing are inseparable in practice. Discovery answers "which instances exist?" Load balancing answers "which instance do I call?" They're two steps of the same decision.

The registry returns a list of healthy instances. Something must pick one. The strategies:

Round-robin: cycle through instances in order. Simple, distributes load evenly when all instances have equal capacity and requests have equal cost. Breaks down when requests are wildly different in cost.

Least-connections: send the next request to the instance with the fewest active connections. Better for variable-cost workloads.

Consistent hashing: send requests with the same key to the same instance (see Q34). Useful when you want session affinity or locality (e.g., cache-local routing in distributed caches).

Random: pick an instance at random. Surprisingly effective — it avoids the thundering herd problem of all clients synchronously choosing the "least loaded" instance at the same moment.

In a service mesh, the sidecar proxy handles this automatically, with configurable policies per service. In client-side discovery, each client library implements it.

The mental model

Hardcoded addresses fail because:
  Services restart on new IPs
  Instances scale up/down dynamically
  Rolling deployments create temporary multi-version coexistence
  Hosts are replaced during maintenance

Service Registry = live database of running instances
  Registration: service starts → registers {name, IP, port}
  Deregistration: service stops → unregisters (or TTL expires)
  Health checks: registry pings instances; unhealthy → removed from results
  Result: clients always get a live, current list of instances

Discovery patterns:
  Client-side: client queries registry → picks instance → calls directly
    Pros: simple infra, full client control
    Cons: every client in every language reimplements discovery + load balancing

  Server-side (proxy/service mesh): client calls fixed address →
    proxy queries registry → picks instance → forwards
    Pros: clients are dumb, logic in one place, language-agnostic
    Cons: extra hop, more infrastructure

Kubernetes approach:
  Service → stable DNS name → kube-proxy load balances to healthy pods
  Client just uses hostname; discovery is transparent

Discovery + load balancing are inseparable:
  Discovery: which instances are available?
  Load balancing: which instance do I call right now?
  Round-robin, least-connections, consistent hashing, random

Service mesh (Istio, Linkerd):
  Sidecar proxy injected alongside every service
  Discovery, LB, retries, circuit breaking, mTLS, tracing → all in sidecar
  Application code knows nothing about any of it

Service discovery exists because in a dynamic system, addresses are not constants — they're variables. Any system that hardcodes addresses breaks the moment the environment changes. Discovery makes addresses a runtime lookup rather than a build-time constant, which is the only model that works when services scale, restart, and move continuously.