17 min read

Chapter 1 — The Debugging Method

How Senior Engineers Find Bugs Instead of Guessing

Part 1: Foundations. No prerequisites.


Opening

Watch a junior engineer and a senior engineer debug the same problem and the difference is not the tools. They both have the same DevTools, the same logs, the same editor. The difference is what they do in the first ten minutes.

The junior engineer forms one theory immediately ("it's probably the cache"), changes something, refreshes, and repeats. Sometimes they get lucky. When they do not, they end up an hour later with five changes in the code, no idea which one mattered, and a bug that may or may not be fixed.

The senior engineer does something that looks slower and is much faster. They write down exactly what is wrong, make it happen on demand, gather evidence before having opinions, list several possible causes, and then run the cheapest check that eliminates the most possibilities. Each step makes the space of possible causes smaller. The bug has nowhere to hide.

This chapter teaches that process and the vocabulary that goes with it. Every later chapter is an instrument you plug into this process.


Part 1: Symptom, Cause, Root Cause — Getting the Words Right

Most debugging confusion starts with mixing up three different things.

Symptom. What someone observes. "The checkout page shows a blank screen." "p99 latency jumped to 4 seconds." "Users got charged twice." A symptom is a fact about behaviour, not an explanation.

Cause (or proximate cause). The thing that directly produced the symptom. "The React component threw because order.items was undefined." Fixing only this often makes the symptom go away while the real problem remains.

Root cause. The deepest cause that, if fixed, prevents this whole class of failure. "The API started returning { data: { items } } instead of { items } after a backend refactor, and nothing validates the response shape." Ask "why?" until the answer is something you can change in the system — a missing check, a wrong assumption, a missing test, a process gap. That is the root cause.

Engineers also use these words constantly:

  • Bug / defect — the mistake in the code or configuration. Formally, a mistake by a person introduces a fault (defect) in the code, which under certain conditions produces a failure (the visible wrong behaviour). You will hear all three words used loosely; know that they describe a chain.
  • Trigger — the specific condition that turns a dormant defect into a failure. A date-parsing bug might sit in the code for a year and trigger only on 29 February.
  • Contributing factor — something that did not cause the failure but made it worse or more likely. "Retries without backoff turned one slow database into a full outage."
  • Expected vs actual — the two sentences every bug report needs. "Expected: total is ₹499. Actual: total is ₹4,990." If you cannot write both precisely, you do not understand the bug yet.
  • Regression — something that used to work and now does not. This is great news when debugging, because it means a change caused it, and changes can be found.

Part 2: The Loop

Every disciplined investigation follows the same loop. The steps are simple. The discipline is doing them in order.

Step 1 — Reproduce

Reproduce (repro) means making the failure happen on demand. A bug you can reproduce is half fixed. A bug you cannot reproduce is a rumour.

  • Repro steps — the exact sequence that triggers the failure: which user, which page, which input, which browser, which environment.
  • Repro rate — how often the steps trigger it. "3 out of 10 attempts" is real information: it tells you the bug is timing-, data- or state-dependent.
  • Minimal reproducible example (MRE) — the smallest setup that still fails. You get it by removing things until the bug disappears, then putting back the last thing you removed. Building an MRE often is finding the bug, because the last thing you removed is usually the cause.

If you truly cannot reproduce, you do not stop. You switch from experiments to evidence: logs, traces, error reports, and the exact state of the user who saw it. That is what Part 2 of this track is about.

Step 2 — Observe Before You Theorise

Collect facts before forming opinions. Read the full error message. Read the full stack trace. Open the Network tab and look at the actual response. Check the time the problem started. Check what else happened at that time.

The most common debugging failure is skipping this step: forming a theory from the first five words of an error and then spending an hour proving that theory instead of reading the rest of the message.

Step 3 — Form Several Hypotheses

A hypothesis is a specific, testable statement about the cause: "The response is HTML instead of JSON because the request is being redirected to the login page." Not "something's wrong with auth."

Write down at least three. One hypothesis means you will try to prove it. Three means you will try to tell them apart — which is what actually finds bugs.

Then rank them by two things: how likely each one is, and how cheap it is to check. Check cheap-and-likely first. A two-second check that eliminates a 30%-likely cause beats a twenty-minute check of a 50%-likely one.

Step 4 — Narrow Down

Run one check at a time and let the result eliminate possibilities. The question to ask of every check is: "What will I conclude if this comes back A, and what if it comes back B?" If both answers lead to the same next step, the check is not worth running.

Change one variable at a time. If you change the cache config and the query and the timeout together and the bug disappears, you have learned nothing about which one mattered — and you may have hidden the real cause behind a coincidence.

Step 5 — Fix the Cause, Not the Symptom

A fix should explain the symptom completely. If your fix makes the bug go away but you cannot explain why the bug happened, you have not found it. You have moved it.

Classic symptom fixes that hide bugs:

  • Adding a try/catch that swallows the error.
  • Adding ?. to make undefined stop crashing, when the real question is why the data is missing.
  • Adding a retry around something that fails, when the real question is why it fails.
  • Increasing a timeout, when the real question is why it is slow.

Sometimes these are the right short-term move (see Part 5). They are never the root-cause fix.

Step 6 — Verify

Run the original repro steps and confirm the failure is gone. Then confirm you did not break anything adjacent. Then — if you can — write a regression test: a test that fails on the old code and passes on the new code. That test is proof the fix works and protection against the bug coming back.

Step 7 — Prevent

Ask one last question: what would have caught this earlier? A type, a schema validation, a test, an alert, a lint rule, a code-review checklist item. The best engineers leave the system slightly harder to break than they found it.


Part 3: Narrowing Techniques — How to Cut the Search Space in Half

These are the core moves. Every later chapter is a way to apply one of them to a specific layer.

If something worked at point A and is broken at point B, check the midpoint. Whichever half the bug is in, check its midpoint. Ten checks narrow 1,000 possibilities down to one.

You can bisect almost anything:

  • Commits — git bisect does this automatically between a good commit and a bad one (Chapter 14).
  • Code — comment out half a function, or return early halfway through.
  • Data — the import fails on a 10,000-row file? Try the first 5,000 rows.
  • Config — revert half the changed settings.
  • The request path — does the request reach the load balancer? The app? The database? (Chapter 6.)

Differential diagnosis: "works vs broken"

Find a case that works and a case that fails, and list every difference between them. The cause is in that list.

  • Works for user A, fails for user B → different data, roles, region, feature flags, browser?
  • Works locally, fails in production → different config, env vars, build, data size, network, CDN?
  • Worked yesterday, fails today → what changed between yesterday and today?

This is the single most useful question in debugging, and it is asked as: "What is different?"

"What changed?"

Systems that were working do not break on their own. Something changed. The usual suspects, in rough order of frequency:

  1. A deploy — your code or a dependency.
  2. Config — env vars, feature flags, infrastructure settings.
  3. Data — a new kind of input, a table that grew past a threshold, a malformed record.
  4. Traffic — more load, a new client, a bot, a marketing campaign.
  5. Time — an expired certificate, a cron job, a daylight-saving change, a token that expired.
  6. Dependencies — a third-party API that changed behaviour or went down.

"Nobody changed anything" almost always means "nobody we know of changed anything we were looking at." Chapter 14 is entirely about this.

Instrumentation

When you cannot see what is happening, make it visible. Add a log line, a breakpoint, a metric, a timing measurement. Good instrumentation answers one specific question: "Does execution reach this line, and what is the value of X when it does?"

Read the source

When library behaviour surprises you, read the library's code instead of guessing. It is in node_modules. Ten minutes of reading often beats an hour of trial and error.

Rubber duck debugging

Explain the problem, out loud or in writing, step by step, to someone — or something — that knows nothing. You will often find the bug mid-sentence, at the step you were about to skip because it was "obviously fine." This works because explaining forces you to state assumptions you were not checking.


Part 4: Kinds of Bugs — and What Each Tells You

Naming the behaviour of a bug tells you where to search. Engineers use these terms all the time:

TermBehaviourWhat it suggests
Deterministic bugFails every time with the same inputLogic error. Easiest kind: reproduce, step through, fix.
Intermittent bugFails sometimes with apparently the same inputSomething you are not controlling varies: timing, data, state, load, which server handled the request.
Flaky testA test that passes and fails without code changesShared state between tests, timing assumptions, test order, real network or clock.
Race conditionOutcome depends on the order of concurrent operationsTwo things touch shared state without coordination. See the timelines in PostgreSQL Ch 16.
HeisenbugDisappears when you try to observe itAdding logs or a debugger changes timing. Strong hint of a race condition.
RegressionUsed to work, now does notA change caused it. Find the change — bisect.
Environment-specific bugFails only in one environmentConfig, versions, data, infrastructure differ. Use differential diagnosis.
Data-dependent bugFails only for certain recordsUnusual input: null, empty, unicode, huge, old format. Find the record.
Load-dependent bugFails only under trafficExhausted resource: connections, memory, file handles, rate limits, locks.
Time-dependent bugFails at certain times or after a durationCron, timezone, DST, expiry, a slow leak that takes hours to hit a limit.

When someone says "it's intermittent," the next question is always: "Intermittent with respect to what?" Time of day? Which server? Which user? Which input? Finding the variable that correlates with failure is usually most of the work.


Part 5: Production Mode — Stop the Bleeding First

Debugging in production changes the priority order. When users are affected right now, restoring service comes before understanding the cause.

  • Triage — quickly deciding how bad it is and what to do first. How many users? Which features? Is it getting worse?
  • Severity (SEV1, SEV2, …) — a shared label for how bad an incident is. SEV1 usually means a core feature is down for many users.
  • Blast radius — how much of the system or user base the problem affects.
  • Mitigation — anything that reduces user impact without necessarily fixing the cause: rolling back a deploy, turning off a feature flag, scaling up, failing over, blocking a bad client.
  • Rollback — returning to the last known-good version. Often the fastest mitigation for a regression.
  • Hotfix — an urgent, minimal fix deployed outside the normal release cycle.
  • Workaround — a way for users or operators to avoid the bug while it stays unfixed.

The sequence is: detect → triage → mitigate → find root cause → fix properly → postmortem. Chapter 15 covers this in full. The key habit now: when production is on fire, the first question is "how do we stop users being hurt?", not "what is the bug?".


Part 6: The Traps

These are the ways smart people waste hours.

Confirmation bias. You believe it is the cache, so you only look for evidence that it is the cache. Counter it by asking: "What would I expect to see if I am wrong?" and checking that.

Changing several things at once. You lose the ability to know what mattered. Revert, then change one thing.

Trusting an assumption you never checked. "The env var is definitely set." "That function definitely gets called." The bug is very often hiding in the thing that was "definitely" fine. Verify it — it takes a minute.

Debugging the wrong thing. The browser is showing a cached bundle. The server is running yesterday's build. You are tailing logs from the staging pod, not production. Before debugging, confirm you are looking at the code and environment that actually fails.

Correlation is not causation. Errors went up at 14:00 and a deploy went out at 14:00. That is a strong lead, not proof. Confirm with a rollback or a direct observation.

Not reading the whole error. The answer is often in the second line of the message, or the cause at the bottom of the stack trace (Chapter 2).

Fixing it without understanding it. "I changed this and now it works" is the beginning of an investigation, not the end. If you cannot explain why the change fixes it, the bug may just be hiding.


The Worked Example — The Method in Ten Minutes

Symptom (from a teammate): "The orders page is broken."

1. Make the symptom precise. You open the page. Expected: a list of orders. Actual: a blank page. The Console shows TypeError: Cannot read properties of undefined (reading 'map') in OrdersList.tsx:14.

2. Reproduce. It happens every time, for your account, in production and locally against the production API. Locally against the staging API it works. That is a differential: the production API returns something different.

3. Observe. In the Network tab, the production response is { "data": { "orders": [...] } }. The component reads response.orders.

4. Hypotheses. (a) The backend changed the response shape. (b) A proxy is wrapping responses. (c) The frontend is calling the wrong endpoint version. Cheapest check: look at the backend's recent deploys. A deploy 40 minutes ago changed the serializer. Hypothesis (a) confirmed; (b) and (c) eliminated because staging — which has not received that deploy — returns the old shape.

5. Mitigate. Roll back the backend deploy. The page works.

6. Root cause. The backend changed a response contract with no versioning, and the frontend has no response validation, so the mismatch surfaced as a crash deep in rendering rather than a clear error at the API boundary.

7. Prevent. Validate API responses at the boundary (for example with a schema), so a contract break produces "orders response failed validation: expected orders array" instead of a blank page. Add a contract test between the services.

Notice what did not happen: nobody added ?. to orders?.map and declared victory.


The Causal Chains

Self-test: predict each consequence before revealing it.
  • If you cannot reproduce the bug
  • If the bug is intermittent
  • If a bug disappears when you add logging or a debugger
  • If it worked yesterday and fails today
  • If it works in one environment and fails in another
  • If you change three things and the bug disappears
  • If your fix is a try/catch, a ?., a retry or a bigger timeout
  • If production users are affected right now

The Rule You Keep Forever

Never act on a single theory. Write down several, find the cheapest check that tells them apart, and change one thing at a time. Every step should shrink the space of possible causes — if it does not, stop and rethink.


Vocabulary Map

TermWhat it means
SymptomThe observable wrong behaviour — a fact, not an explanation
Proximate causeThe thing that directly produced the symptom
Root causeThe deepest cause that, once fixed, prevents the whole class of failure
TriggerThe condition that turns a dormant defect into a visible failure
Contributing factorSomething that made the failure worse or more likely without causing it
Expected vs actualThe two precise statements that define a bug
RegressionSomething that worked before a change and fails after it
Repro / repro stepsMaking the failure happen on demand; the exact steps that do it
Repro rateHow often the repro steps trigger the failure
Minimal reproducible example (MRE)The smallest setup that still shows the failure
HypothesisA specific, testable statement about the cause
BisectionHalving the search space repeatedly — commits, code, data, config, path
Differential diagnosisComparing a working case with a failing case and examining the differences
InstrumentationAdding logs, metrics, breakpoints or timers to make behaviour visible
Rubber duck debuggingExplaining the problem step by step to expose unchecked assumptions
Intermittent / flakyFails only sometimes; flaky usually refers to tests
HeisenbugA bug that disappears when observed, usually because observation changed timing
Race conditionOutcome depends on the uncontrolled ordering of concurrent operations
TriageQuickly deciding severity and what to do first
Severity (SEV)A shared label for incident impact, SEV1 being the worst
Blast radiusHow much of the system or user base is affected
MitigationReducing user impact without necessarily fixing the cause
RollbackReturning to the last known-good version
HotfixAn urgent minimal fix shipped outside the normal release cycle
WorkaroundA way to avoid the bug while it remains unfixed
Regression testA test that fails on the buggy code and passes on the fixed code

Practice Problems

Problem 1 — Make it precise. A product manager writes: "Search is broken on mobile." Write the five questions you would ask before opening any code, and say what each answer would rule in or out.

Problem 2 — Rank the hypotheses. Since this morning, about 5% of image uploads fail with a generic "Upload failed" message. Uploads under 1 MB always work. Write four hypotheses, estimate how likely and how cheap to check each one is, and say which you check first and why.

Problem 3 — Spot the symptom fix. A teammate fixes Cannot read properties of null (reading 'email') in the profile page by changing user.email to user?.email ?? ''. The crash stops. What question should you ask in code review, and what might the real bug be?

Problem 4 — Intermittent with respect to what? A background job fails roughly once an hour with no obvious pattern. List five variables you would correlate the failures against, and what a correlation with each would suggest.

Problem 5 — Production mode. Error rates jump from 0.1% to 12% three minutes after a deploy. Your teammate wants to read the diff to find the bug. What do you do first, and why?


What's Coming Next

The method is useless if you cannot read the evidence. Chapter 2 starts with the most common evidence of all: the error message and the stack trace. You will learn to read them literally — every word in TypeError: Cannot read properties of undefined (reading 'map') is telling you something specific — and to find the one frame in a forty-line stack trace that actually matters.