Part II — Debugging Deterministic Software

The function that reads correctly and runs wrong

Chapter 5 convicted the handoff. Now you open the convicted function, and it looks innocent:

# billing/discounts.py
def apply_discount(order):
    rate = RATE_TABLE[order["tier"]]       # line 12
    total = order["subtotal"] * (1 - rate) # line 13
    if order.get("coupon"):                # line 14
        total -= COUPONS[order["coupon"]]  # line 15
    return round(total, 2)                 # line 16

A premium-tier order for $200 with a $20 coupon returns $160 instead of the expected $140. Read the code: line 12 looks up the tier rate (0.2 for premium), line 13 computes $160, lines 14–15 subtract the coupon. The logic reads correctly. Two engineers stare at it for twenty minutes and conclude “the coupon table must be stale” — then spend an afternoon auditing COUPONS, which is fine.

OBSERVATION: apply_discount({"tier": "premium", "subtotal": 200, "coupon": "SAVE20"}) returns 160.0; spec expects 140.0 on this code version, this input. HYPOTHESIS H1 (stale table): COUPONS["SAVE20"] holds 0 instead of 20 at runtime. HYPOTHESIS H2 (skipped branch): line 14’s condition is falsy at runtime, so line 15 never executes. INFERENCE: none yet — code reading predicts both H1 and H2 produce nothing observable without runtime values.

Chapter 5 gave you the frame. This chapter’s question: when the code reads correctly, what live values prove which line actually diverged?

Why reading code fails here

Code reading simulates an idealized machine: you substitute the values you believe each variable holds. Three failure modes make that simulation lie:

1. Believed values vs. runtime values. You read order.get("coupon") as “the coupon string,” because the fixture in your head has one. At runtime, the caller passes "coupon": "save20" (lowercase) or coupon: None after a normalization step you never opened. The branch behaves perfectly — on values you never checked.
2. Print-and-glance. The engineer adds print(total) at line 16, sees 160.0, and concludes “line 13 is wrong.” A single end-of-function print cannot separate H1 (wrong table value, branch ran) from H2 (branch skipped). The print confirmed the symptom and masqueraded as localization.
3. Debugger tourism. Stepping through with pdb but looking without predicting — watching values scroll past and nodding at each one. Without a written prediction per line, every observed value looks “plausible” in hindsight, and the actual divergence (coupon is None, rate is 0.2 — correct) slides by unflagged. This is confirmation bias with a tool attached: developers tend to seek evidence that fits the current theory rather than evidence that would break it, and training in explicit hypothesis testing measurably reduces the effect (Çalıklı & Bener, 2013).

The discipline this chapter installs: predict the value at each line before you look, then let the runtime overrule you. A prediction that survives inspection is evidence. An inspection without a prediction is a tour.

The method: predict-then-inspect with watch invariants

Stop the program at the convicted handoff and work in this order:

1. Write predictions first. For each line in the suspect region, write the value you expect each variable to hold before running. Include types, not just magnitudes: "SAVE20" (str) vs. None vs. missing key behave differently at line 14.
2. Break at the handoff, not the symptom. breakpoint() (or pdb.set_trace()) goes at the entry of the convicted function — line 12, not line 16. Inspecting the output tells you it is wrong; inspecting the entry tells you which assumption broke.
3. Watch invariants, not just values. State Chapter 2’s checkpoints as live assertions: at line 12, order["tier"] in RATE_TABLE; at line 14, type(order.get("coupon")); at line 16, total <= order["subtotal"]. Check each invariant at its line. The first violated invariant is the first divergence, restated in live state.
4. One probe per hypothesis. H1 predicts COUPONS["SAVE20"] == 0 at line 15. H2 predicts line 15 never executes. A single breakpoint at line 15 with condition-free inspection separates them: if you never hit line 15, H2 is convicted regardless of the coupon table’s contents.

    flowchart TD
    B[breakpoint at function entry] --> W["write per-line predictions: value AND type"]
    W --> S[step forward, compare each line to its prediction]
    S --> L15{"line 15 executed?"}
    L15 -->|no| H2["H2: branch skipped — inspect the caller that made the condition falsy"]
    L15 -->|yes| V{"value used matches prediction?"}
    V -->|no| H1["H1: wrong value used — inspect the table/data that supplied it"]
    V -->|yes| NEXT[keep stepping to the first falsified prediction]
  

Minimal tooling, deliberately so:

# billing/discounts.py — instrumented for inspection, not patched
def apply_discount(order):
    breakpoint()  # entry: predict order, RATE_TABLE, COUPONS before looking
    rate = RATE_TABLE[order["tier"]]
    total = order["subtotal"] * (1 - rate)
    if order.get("coupon"):
        total -= COUPONS[order["coupon"]]
    return round(total, 2)

# pdb session (predictions written BEFORE running):
# P12: order["tier"]=="premium", rate==0.2
# P14: order.get("coupon")=="SAVE20" (truthy) → branch runs
# P15: COUPONS["SAVE20"]==20 → total==140.0
(pdb) p order
{'tier': 'premium', 'subtotal': 200, 'coupon': None}   # P14 FALSIFIED
(pdb) p RATE_TABLE[order["tier"]]
0.2                                                     # P12 confirmed
# line 15 never hit → H1 (stale table) suspended; H2 (skipped branch) supported

OBSERVATION (constructed illustration): order["coupon"] is None at runtime; rate is 0.2 as predicted; line 15 never executes. UPDATED BELIEF: H2 supported — the caller normalized the coupon away before entry. H1 suspended: the table was never consulted, so its contents are irrelevant to this failure. INFERENCE: the first divergence is above this function, at the caller that set coupon to None. The fix belongs there, not in the discount table.

Print discipline has its place: when a debugger is unavailable (remote job, tight loop), emit repr() with types at handoff lines — print("L14", repr(order.get("coupon")), type(order.get("coupon"))) — never a bare value at the output. But prints are frozen after the run; the debugger lets you follow the falsified prediction upward into the caller in the same session.

When the divergence is one iteration out of thousands and the input will not minimize (the bug needs the volume), do not step through every pass. Trigger the breakpoint on the failing case: breakpoint() guarded by if order["id"] == 8814, or pdb’s conditional form (break discounts.py:15, order["coupon"] is None). The prediction sheet is unchanged — you are still writing expected values before looking — you have just skipped the interior of the regime to land on the seam, which is the same move Chapter 7 makes with inputs.

A note on what practitioners actually do. Beller and colleagues instrumented 458 developers’ IDE usage over the equivalent of ten work-years and surveyed 176 more, and found that print-statement debugging dominates: most developers rarely set breakpoints and largely avoid the debugger’s richer features (Beller et al., 2018). The lesson this chapter draws is not “use the debugger more.” It is that the predict-then-inspect discipline is what matters, and it works with disciplined prints too — the tool is secondary, the written prediction is not. Reach for the debugger when you need to walk upward interactively; otherwise instrument prints, but write the predictions first either way.

Demonstration: the afternoon wasted on the wrong table

Reconstruct the misdiagnosis so you can feel its pull. The team read lines 12–16, judged them correct, and hypothesized the data (COUPONS stale) rather than the state (coupon absent). They ran grep SAVE20 config/coupons.yaml, found SAVE20: 20, concluded “table is fine, must be caching,” and restarted services. Two more hours. A predict-then-inspect session would have taken six minutes: prediction P14 written, p order observed, H2 convicted, attention redirected to the caller — where a normalization refactor (coupon.strip().upper() replaced by a lookup returning None on miss) had silently erased the coupon one call up the stack.

The pairing with Chapter 5 is exact: the traceback names the function; only live state names the line. And the same-symptom/different-cause discipline carries over — $160 instead of $140 arises identically from H1 (table holds 0, branch runs, 200*0.8 - 0 = 160) and H2 (table holds 20, branch skipped, 200*0.8 = 160). Output values cannot separate them. Entry state can, in one inspection.

Research lineage: hypotheses beat locations

The strongest support for this chapter’s method is a controlled experiment by Alaboudi and LaToza. Twenty developers debugged while recording their hypotheses. Three findings matter here. Developers formed few hypotheses — about two per defect — so the plural-hypothesis rule (H1 and H2) is asking for something people do not do naturally. Having a correct hypothesis early strongly predicted eventual success. And when the researchers gave participants candidate fault locations it did not help them, whereas giving them candidate hypotheses made them roughly six times more likely to succeed (Alaboudi & LaToza, 2020). That is the empirical case for this whole chapter: a convicted line (Chapter 5) is a location; what turns it into a fix is a pair of competing hypotheses tested against live state. Their follow-up tool, Hypothesizer, operationalizes exactly the prediction sheet this chapter uses (Alaboudi & LaToza, 2023).

“Forming about two hypotheses per defect” and “touring the debugger without predicting” are the same failure the diagnostic-error literature calls premature closure — settling on the first plausible account before the alternatives are tested — which Chapter 47 treats as a named, procedural (not educational) problem, and which is also the failure mode of the fast pattern-recognition that Chapter 57 shows experts rely on under time pressure. The written prediction sheet is the countermeasure: it forces a second hypothesis into existence and gives the runtime something specific to falsify.

The “skipped branch” hypothesis (H2) has its own lineage. Ko and Myers built the Whyline, which lets a programmer ask “why did this happen?” and “why didn’t this happen?” directly about program output; in a controlled study the Java Whyline cut debugging time by roughly a factor of eight and let participants complete about 40% more tasks (Ko & Myers, 2008). “Why didn’t line 15 execute?” is a why-didn’t question, and the answer — order["coupon"] is None at entry — is exactly the runtime event the Whyline surfaces. Where a Whyline-style tool is not available, the entry breakpoint plus a written “P15: hit” prediction is the manual substitute.

Lab 6: predict-then-inspect

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own prediction sheet.

Setup. Take a 5–15 line function with a wrong-output bug (use apply_discount with the None-coupon caller, or inject an analogous defect: a falsy-but-valid value — 0, "", None, [] — that silently skips a branch). Pin the repro: input, code hash, expected vs. observed output.

Task.

1. Write H1 (wrong value used) and H2 (correct value never reached / branch skipped) with distinct line-level predictions before running anything. Name the exact line each hypothesis convicts.
2. Place one breakpoint at function entry. Independent variable: inspection point (entry → line-by-line). Controlled variables: input, code version, environment.
3. At each line, record three columns: PREDICTION (value + type), OBSERVATION (p output verbatim), VERDICT (confirmed/falsified). Stop at the first falsified prediction — that line is the first divergence.
4. Step up (up in pdb) into the caller and record where the falsified value originated.

Success criterion. A prediction sheet with at least one falsified prediction, the convicted line named, and the caller frame where the bad value originated. An inspection log without prior predictions is not completing the lab — predictions are what make the observation falsifying rather than decorative.

Companion tool: State Inspector

What it accepts: the convicted handoff from Chapter 5, the written predictions per line (value + type), and the observed p outputs with the pdb transcript or disciplined prints. What it performs: it checks completeness — every line in the suspect region has a prediction before its observation, every observation records repr() plus type, and the first falsified prediction is marked as the divergence point with the upward up-frame origin. What it can establish: which line first diverged from intent under this reproduction, and whether the profile is wrong-value (H1: line ran with bad data) or skipped-computation (H2: line never ran). What it cannot establish: why the caller produced the bad value (that is a further upward inspection, possibly reaching Chapter 9’s environment layer), nor correctness beyond this repro — passing state on one input does not certify the function. How its output changes your next action: a wrong-value verdict routes you to the data/table that supplied the value; a skipped-computation verdict routes you to the caller that shaped the condition. Either way, the next step is named as a frame and line, not a vague “look upstream.”

Paper form, sufficient for this chapter:

Entry breakpoint: file:line (fn) | input hash: ___
Line | PREDICTION (repr+type) | OBSERVATION (repr+type) | VERDICT
___  | ___                   | ___                     | ___
FIRST FALSIFIED LINE: ___  ORIGIN (up-frame): ___
PROFILE: wrong-value / skipped-computation   EVIDENCE: ___

Where a software implementation does not yet exist in the reader’s stack, this sheet is the tool. The prediction discipline precedes any automation.

Reusable procedure: never debug blind again

1. Take the convicted handoff from the Traceback Inspector (Chapter 5) as the inspection target.
2. Write predictions for every line in the region: value, type, and branch taken/not-taken.
3. Break at entry, inspect forward; mark the first falsified prediction.
4. Step up into the caller to name the value’s origin.
5. File the State Inspector sheet with transcript; convert the falsified invariant into a Chapter 8 assertion.

Failure modes

• Touring without predicting. Stepping through the debugger nodding at values. Hindsight makes every value look inevitable; only a prior prediction can be falsified.
• Output-only printing. print(total) at the return line. Confirms the symptom, localizes nothing; H1 and H2 predict identical outputs here.
• Value without type. Recording coupon is empty instead of None vs. "" vs. missing. At line 14 all three are falsy but originate from different callers and demand different fixes.
• Fixing at the inspection point. Patching apply_discount to handle None when the caller broke its contract. The divergence is in the caller; the guard here would be Chapter 4’s silencing anti-pattern.
• Agreement-as-evidence. “Two of us read the code and agree the table is stale.” Agreement is not observation; runtime values overrule readers.

Limits, per contract: one inspection convicts a line under one repro; it does not prove the function correct, does not rule out a second defect on other inputs, and does not replace human verification where money, safety, or production traffic is at stake. UNKNOWN where the debugger cannot reach (remote, optimized, non-reproducible runs get disciplined prints instead).

References

• Abdulaziz Alaboudi and Thomas D. LaToza. Using Hypotheses as a Debugging Aid. Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2020. https://doi.org/10.1109/VL/HCC50065.2020.9127273
• Abdulaziz Alaboudi and Thomas D. LaToza. Hypothesizer: A Hypothesis-Based Debugger to Find and Test Debugging Hypotheses. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), 2023. https://doi.org/10.1145/3586183.3606781
• Andrew J. Ko and Brad A. Myers. Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior. Proceedings of the 30th International Conference on Software Engineering (ICSE), 2008, pp. 301–310. https://doi.org/10.1145/1368088.1368130
• Moritz Beller, Niels Spruit, Diomidis Spinellis, and Andy Zaidman. On the Dichotomy of Debugging Behavior Among Programmers. Proceedings of the 40th International Conference on Software Engineering (ICSE), 2018, pp. 572–583. https://doi.org/10.1145/3180155.3180175
• Gül Çalıklı and Ayşe Başar Bener. Influence of Confirmation Biases of Developers on Software Quality: An Empirical Study. Software Quality Journal 21(2), 2013, pp. 377–416. https://doi.org/10.1007/s11219-012-9180-0

Debugging Checklist

• Convicted handoff from Chapter 5 identified as the inspection target?
• Per-line predictions (value + type + branch) written before running?
• Breakpoint at function entry, not at the symptom line?
• Observations recorded as repr() + type, verdict per line?
• First falsified prediction marked; origin found via up-frame?
• H1 (wrong-value) vs. H2 (skipped-computation) separated by whether the line executed?
• No code-reading consensus, bare print, or single output treated as localization?

What This Chapter Established

• Predict-then-inspect: written per-line predictions (value + type) falsified or confirmed by live debugger state, with watch invariants marking the first divergence.
• Empirical backing: candidate hypotheses help debugging ~6x where candidate fault locations do not (Alaboudi & LaToza); “why didn’t” questions map to the skipped-branch profile (Ko & Myers); print debugging dominates practice, so the discipline must be tool-agnostic (Beller et al.); debugger tourism is confirmation bias (Çalıklı & Bener).
• The apply_discount demonstration: identical wrong output ($160) from a stale table (H1) vs. a skipped branch on None coupon (H2), separated by one entry breakpoint — constructed illustration, no measured runs claimed.
• Lab 6 as a proposed prediction sheet the reader executes; the State Inspector contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: function correctness beyond this repro, caller-layer causes above the inspected frame, or any claim resting on reading consensus or output-only prints.

Next

Live state convicts the line that diverged on this input — but some defects diverge only on inputs nobody thought to run. The function above passes every “normal” test with a valid coupon and fails only at the edges the suite never covers. The next chapter hunts where deterministic bugs cluster: the boundaries.