Part VIII — Building the AI Debugger

The diagnosis that survived everything except verification

Chapter 48 split the space to one survivor — but survival is not proof. A practitioner holds H1 (stale snapshot) as the last branch standing for the fabricated citation: the chunk was absent, expansion changed nothing, wording swaps changed nothing. She ships the re-index, the symptom disappears for a week, and the postmortem records “root cause: stale index.” A month later the fabrication returns with a fresh index. The original evidence never distinguished “stale snapshot caused it” from “stale snapshot accompanied it” — no prediction-match ledger, no independent verification evidence, no human sign-off on the causal claim. The branch survived; the diagnosis was never verified.

OBSERVATION: H1 survived elimination; the re-index coincided with symptom relief once; no pre-registered verification predictions, no independent evidence stream, no reversal test recorded. HYPOTHESIS H1 (causal): the stale snapshot caused the fabrication. H2 (accompaniment): staleness correlated while another factor (e.g., chunking rule, ingestion filter) caused it. H3 (multi-cause): staleness plus a second factor jointly required. INFERENCE: none yet — H1/H2/H3 predict different reversal and independence signatures: H1 predicts reintroducing staleness reintroduces the symptom; H2 predicts it does not; H3 predicts partial return.

This chapter’s question: what verification bar turns a surviving hypothesis into a signed diagnosis — and what evidence counts as independent enough to clear it?

Why “it worked after the fix” fails first

The obvious move — citing post-fix relief as proof — fails because relief is a downstream symptom, and downstream symptoms never diagnose (book-wide hard rule). Five insufficiencies hide behind relief:

1. Coincident relief. The re-index shipped with a config refresh, a restart, and a prompt tweak. Which intervention caused relief is unassigned — multi-variable disposal proves nothing.
2. Single-run confirmation. One good week cited as verification. Nondeterministic systems need repeated confirmation runs plus a reversal (reintroduce → symptom returns → remove → relief returns) before causal language.
3. Dependent re-evidence. “Verifying” with the same log, same assistant, same metric that produced the hypothesis. Verification evidence must be independent of discovery evidence — different stream, different tool, or different operator.
4. No ledger. Predictions scattered across chat, runs, and memory. Without a prediction-match ledger (prediction → outcome → match/mismatch, dated), nobody can audit what was actually confirmed. This is the postdiction-as-prediction error: Nosek and colleagues frame it as the core threat to credibility — generating a hypothesis from existing observations and then presenting it as if it had been tested against new ones (Nosek et al., 2018). Kerr named the specific move HARKing — hypothesizing after the results are known — and its sub-types; the one that dominates incident docs is constructing a “root cause” story to fit the relief (Kerr, 1998). But HARKing is about the hypothesis; Gelman and Loken showed the same false “verified” can come from the analysis choices made after seeing the runs — which trials to count, which metric, which match threshold — even with an honest pre-stated hypothesis and no p-hacking (Gelman & Loken, 2013). So the dated ledger is a preregistration of the whole plan: not just the hypothesis, but the observable, the trial count, and the match criterion. It makes it impossible to pretend a post-hoc match was foreseen — or a post-hoc analysis path.
5. Unsigned high-impact claims. Causal verdicts affecting production, refunds, or access shipped on assistant authority. High-impact diagnoses require human verification against raw records — always, including here.

OPINION: most “root causes” in incident docs are last-hypotheses-standing wearing verification costumes. A diagnosis without a ledger is a rumor with a timestamp.

The incident-analysis field has been saying a version of this for years. Cook’s How Complex Systems Fail argues that catastrophe in a complex system requires multiple contributing failures, so there is rarely an isolated root cause to find (Cook, 2000). Allspaw’s Blameless PostMortems and Dekker’s Field Guide to Understanding ‘Human Error’ both warn that naming a single “root cause” after the fact is usually hindsight bias — the counterfactual “if only X” describes a world that did not exist (Allspaw, 2012; Dekker, 2014). Leveson’s STAMP formalizes the point: a complex failure comes from inadequate enforcement of constraints across the whole control structure, not one broken component, so “find the root cause” is often the wrong frame (Leveson, 2011); Chapter 59 develops its analysis method, CAST. That is precisely why this chapter carries H3 (multi-cause) as a live hypothesis and demands a reversal test: to distinguish “stale snapshot was one condition among several” from “stale snapshot was the cause.”

The mental model: verification is a trial with a transcript. The diagnosis is the charge; the ledger is the transcript; independent evidence is the second witness; the human sign-off is the verdict. Surviving elimination gets the case to court. Only prediction-matches on independent evidence, including a reversal where feasible, convict. That ledger is not a new artifact — it is the Chapter 45 diagnostic-case record with the hypothesis-and-verification fields filled: same object, later stage.

The method: the verification bar in four parts

A signed diagnosis requires all four, recorded in the ledger:

1. Prediction-match ledger. Every live hypothesis’s pre-dated predictions listed with outcomes and match/mismatch verdicts. Minimum to convict: the winner matches on all its verification predictions; every loser mismatches on at least one (exoneration cited to its deciding run).
2. Independence of verification evidence. At least one confirming observation from a stream not used to generate the hypothesis: different log, different probe, different operator re-running from the bundle, or the assistant’s proposal confirmed by human-executed runs. Same-stream confirmation is corroboration theater.
3. Reversal test (where feasible). Reintroduce the convicted condition and observe symptom return (×3 trials where nondeterministic), then remove and observe relief. Where reversal is unsafe or impossible (production harm, destructive state), record the infeasibility explicitly and cap the verdict at PROVISIONAL — never upgrade by enthusiasm.
4. Human sign-off by impact. Low-impact (notebook note, draft config): practitioner sign-off on the ledger suffices. High-impact (production change, customer correction, access/refund action): a human verifies the deciding records raw (hashes, verbatim lines, trial counts) and signs name + date. No assistant self-sign-off, ever.

    flowchart TD
    W["surviving hypothesis from Ch48 elimination"] --> P{"every live hypothesis's PRE-DATED predictions matched to outcomes?"}
    P -->|"no / post-dated"| UNS["UNSUPPORTED — reopen the space, not the production change"]
    P -->|yes| L{"every loser exonerated with its deciding run cited?"}
    L -->|no| UNS
    L -->|yes| I{">=1 confirming observation from an INDEPENDENT stream / operator / tool?"}
    I -->|no| PROV["PROVISIONAL — same-stream confirmation is theater; name the missing part"]
    I -->|yes| RV{"reversal: reintroduce the condition -> symptom returns x3, then remove -> relief?"}
    RV -->|"infeasible / unsafe"| PROV
    RV -->|"done, matches x3"| SO{"impact class?"}
    SO -->|low| SIGN["SIGNED — practitioner sign-off on the ledger"]
    SO -->|high| HV["human verifies deciding records raw (hashes, verbatim, trial counts), signs name + date"]
    HV --> SIGN
  

PREDICTION-MATCH LEDGER (worked sketch; constructed, not measured):
Case refund-quote-041. Winner H1 stale-snapshot; losers H2/H3.
P1 (H1): gold absent from ranked list ............ OBSERVED absent — MATCH
P2 (H1): k-expansion changes nothing (×3) ........ 0/3 changed — MATCH
P3 (H1): wording-swap changes nothing (×3) ....... 0/3 changed — MATCH
P4 (H1, verification): fresh re-index restores
    citation ×3, INDEPENDENT operator reruns ..... 3/3 — MATCH (indep. stream)
P5 (H1, reversal): reintroduce stale snapshot →
    fabrication returns (×3) ..................... 3/3 — MATCH
H2 exonerated at P1/P2. H3 exonerated at P3.
INDEPENDENCE: P4 by second operator from bundle alone. SIGN-OFF: ___ (human).
RULE: all P1–P5 match + independence + reversal = SIGNED; any missing
part caps at PROVISIONAL (record which).

OBSERVATION (constructed illustration, not a measured run): the ledger above matches on all five with independent confirmation and reversal. UPDATED BELIEF: H1 verified for this instance under this bundle and version set; H2/H3 exonerated here — licensed claim only, no generality; without P4/P5 the same table would read PROVISIONAL, not SIGNED.

No model explanation of why the fix worked, no confidence score on the diagnosis, no score deltas, no correlation statistics alone, no single confirming run, no second assistant’s agreement, and no downstream quiet (“tickets dropped”) substitutes for ledger matches on independent evidence with reversal. Matches, witnesses, and signatures decide.

Example: capping an honest PROVISIONAL

Contrast the constructed SIGNED case with a reversal-infeasible one: the convicted condition is a production permission escalation that cannot be reintroduced safely. The practitioner records P1–P4 matches with independent confirmation, marks P5 INFEASIBLE with the reason (“reintroducing grants live access”), and signs PROVISIONAL with the cap stated: “treat as cause for prevention purposes; do not cite as proven cause in external reports.” The cap is not failure — it is the method working. PROVISIONAL with a named missing part beats SIGNED on theater.

Research lineage: preregister the prediction, reverse the condition

Independence of verification evidence is the prediction/postdiction line. Nosek’s distinction is not just about timing — it is that a hypothesis built from a body of evidence has already “used up” that evidence, and confirming it against the same evidence tells you nothing new (Nosek et al., 2018). The requirement for a second stream, a second operator, or human-executed runs confirming an assistant’s proposal is that principle operationalized.

The reversal test is an intervention, not a re-observation. Reintroducing the convicted condition and watching the symptom return is Causal Testing (Chapters 1 and 41) and Zeller’s cause-effect-chain logic: a difference is promoted to a cause only when changing it changes the outcome, in both directions where feasible. Post-fix relief is a single-direction, single-trial observation; the reversal makes it a two-direction, repeated intervention.

PROVISIONAL is the honest verdict when the complex system won’t let you reverse. Cook and Allspaw’s point cuts both ways: sometimes you genuinely cannot isolate one cause because the failure had several, and sometimes you cannot safely reverse the one you found. Both cases end at PROVISIONAL with the reason named — which is the method working, not failing.

Lab 49: ledger one of your own past diagnoses (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own ledger.

Setup. Take one past diagnosis you recorded as “root cause” (any domain). The review regime (memory/impression vs. this chapter’s four-part ledger) is the independent variable; the case materials are controlled. A second reader checks independence.

Task.

1. Before opening materials, write H1/H2/H3 about your diagnosis with distinct predicted ledger outcomes: H1: “ledger shows all four parts present (SIGNED stands)”; H2: “ledger shows missing independence or reversal (downgrade to PROVISIONAL)”; H3: “ledger shows a loser’s exoneration uncited (verdict UNSUPPORTED).”
2. Reconstruct P1–Pn with dates (or mark post-dated), identify the independence stream (or its absence), attempt/justify-no reversal, and obtain sign-off.
3. Record the final verdict: SIGNED / PROVISIONAL (missing part named) / UNSUPPORTED.

Success criterion. A filled ledger with dated predictions, independence stream named, reversal attempted or infeasibility justified, and a signed verdict with caps stated. A re-asserted “root cause” without the ledger is explicitly not completion.

Companion tool: Diagnosis Validation Protocol Runner

What it accepts: the hypothesis space with dated predictions, the execution/trial logs, the independence-stream pointer, and the impact class. What it performs: it matches predictions to outcomes row by row, checks that every loser’s exoneration cites its deciding run, verifies the independence stream differs from discovery, evaluates reversal (done / infeasible-with-reason), and emits SIGNED / PROVISIONAL / UNSUPPORTED with missing parts named. What it can establish: whether a diagnosis meets the verification bar — for the examined case only. What it cannot establish: cross-case generality, future stability under version change, or the absence of unlisted causes (coverage belonged to Ch47). It never treats explanations, scores, correlations, single runs, agreement, or symptom relief as verification. How its output changes your next action: SIGNED → convert to prevention artifact (Part IX); PROVISIONAL → ship prevention with the cap written on it; UNSUPPORTED → reopen the space, never the production change.

Paper form, sufficient for this chapter:

Case ___ Bundle ___ Impact: low / high
LEDGER (prediction → outcome → match?): P1 ___ P2 ___ P3 ___ P4 ___ P5 ___
LOSERS exonerated (run cited): ___  INDEPENDENCE stream: ___
REVERSAL: done (×3 ___) / INFEASIBLE (reason ___)  SIGN-OFF (human, date): ___
VERDICT: SIGNED / PROVISIONAL (missing ___) / UNSUPPORTED

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. No ledger, no license to claim cause.

Reusable procedure: verify every diagnosis the same way

1. Ledger predictions — dated before runs; match each to its outcome.
2. Exonerate losers — every non-winner cites its deciding run.
3. Confirm independently — a second stream, tool, or operator re-runs from the bundle.
4. Reverse where feasible — reintroduce ×3 or justify infeasibility and cap.
5. Sign by impact — human on raw records for high-impact; name and date recorded.

Failure modes

• Relief-as-proof. “Fixed, therefore diagnosed.” Relief is a symptom; ledgers are proof.
• Self-confirming streams. Verifying with the discovery evidence. Independence or theater — no third option.
• Undated predictions. Matches claimed without timestamps. Post-dated matches are inadmissible.
• Skipped reversal, silent. No reversal, no infeasibility note, full causal language anyway. Missing parts cap verdicts aloud.
• Assistant-signed verdicts. The proposer signing its own case. Humans sign; assistants propose.
• Generality creep. “Verified here” slides into “verified everywhere.” Licenses cover the examined bundle and version set — nothing wider.

Limits, per contract: one verdict covers one case under one bundle/version set; it warrants the causal claim stated, not neighboring claims; any later version roll, re-index, or config change bounds the license until re-verified. UNKNOWN wherever predictions post-date outcomes or independence is absent.

References

• Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. The Preregistration Revolution. Proceedings of the National Academy of Sciences 115(11), 2018, pp. 2600–2606. https://doi.org/10.1073/pnas.1708274114
• Norbert L. Kerr. HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review 2(3), 1998, pp. 196–217. https://doi.org/10.1207/s15327957pspr0203_4
• Richard I. Cook. How Complex Systems Fail. Cognitive Technologies Laboratory, University of Chicago, 2000. https://how.complexsystems.fail/
• John Allspaw. Blameless PostMortems and a Just Culture. Etsy Code as Craft, 2012. https://www.etsy.com/codeascraft/blameless-postmortems/
• Sidney Dekker. The Field Guide to Understanding ‘Human Error’ (3rd ed.). Ashgate / CRC Press, 2014. https://www.taylorfrancis.com/books/mono/10.1201/9781317031833/field-guide-understanding-human-error-sidney-dekker
• Andrew Gelman and Eric Loken. The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No “Fishing Expedition” or “p-Hacking” and the Research Hypothesis Was Posited Ahead of Time. Working paper, Department of Statistics, Columbia University, 2013. https://sites.stat.columbia.edu/gelman/research/unpublished/p_hacking.pdf
• Nancy G. Leveson. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press, 2011. https://mitpress.mit.edu/9780262533690/engineering-a-safer-world/

Debugging Checklist

• Every prediction dated before its run, matched to an outcome?
• Winner matches all verification predictions; losers exonerated with cited runs?
• At least one confirming observation from an independent stream/operator?
• Reversal done ×3, or infeasibility justified with a PROVISIONAL cap?
• No forbidden sources (explanation/scores/correlation/single-run/agreement/symptom) as verification?
• Verdict recorded SIGNED / PROVISIONAL / UNSUPPORTED with missing parts named?
• Human sign-off on raw records for high-impact (name + date)?
• License scope written (this bundle, these versions, no wider)?

What This Chapter Established

• The verification bar: prediction-match ledgers, independence of verification evidence, reversal-or-cap, and human sign-off by impact — demonstrated on the constructed SIGNED ledger plus the honest-PROVISIONAL contrast, no measured runs claimed.
• The trial-with-transcript mental model and the relief-is-not-proof rule.
• Lab 49 as a proposed past-diagnosis re-ledger the reader executes; the Diagnosis Validation Protocol Runner contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any cause beyond the examined instance, any generality across versions, or any exemption from re-verification after change. One bar set; nothing universal.
• Research grounding: “prediction-after-outcome” is postdiction-as-prediction / HARKing (Nosek et al.; Kerr — the incident-doc common sub-type is constructing the root-cause story), and analysis choices made after seeing the data are a second route to a false “verified” even with an honest hypothesis (garden of forking paths — Gelman & Loken), so the dated ledger preregisters the whole plan — observable, trial count, match criterion; independence of verification evidence is the prediction/postdiction line — confirming against the discovery evidence tells you nothing new; the reversal test is a two-direction intervention (Causal Testing / Zeller); and PROVISIONAL is the honest verdict when a complex failure has several contributors (Cook; Allspaw; formally, STAMP — Leveson) or the condition cannot be safely reversed.

Next

Individual diagnoses can now be verified — but a debugger that verifies one case well may still fail the next fifty. Case skill and benchmark skill are different claims, and only the second can be stated quantitatively. Chapter 50, “AIDebugBench,” designs the benchmark that would measure debugging skill across task families — its tasks, rubrics, and contamination controls — while reporting no scores, because none have been run.