Part V — Debugging AI-Assisted Development and Research

The literature review reads perfectly — and one paper doesn’t exist

Chapter 27 priced the design. Now its “related work” section summarizes three papers with DOIs, quotes, and confident synthesis — one citation resolves to an unrelated paper, one has no retrievable byte anywhere, and the third says the opposite of what is claimed. The prose is flawless. The provenance is fiction.

OBSERVATION: three citation-shaped strings in the generated review; retrieval logs show zero source documents fetched with matching hashes, and one DOI resolves to a different title. HYPOTHESIS H1 (fabricated reference): the citation string was generated without any retrieval — no source byte supports it. H2 (misattributed source): a real document was retrieved but the claim contradicts its content — present but misread. H3 (synthesis overreach): sources are real and correctly cited, but the conclusion drawn across them exceeds what any combination states. INFERENCE: none yet — H1/H2/H3 predict different provenance rows and are separable only by resolving every reference to a retrieved byte.

This chapter’s question: for each research claim, what retrieved byte supports it — and what happens to claims with none?

Why “check the references look right” fails first

The obvious move — scanning titles and DOIs for plausibility — fails because fabrication is optimized for plausibility. Four defects hide behind citation shape:

1. Typography-as-provenance. Bracketed numbers, author names, and venues look like evidence (Chapter 3). Without byte-level linkage to retrieved content, they are formatting.
2. Title-truth confusion. A real paper with a real DOI cited for a claim it never makes. Existence verification alone (the DOI resolves!) misses misattribution entirely.
3. Synthesis laundering. Three faithfully cited facts combined into a conclusion none supports. Per-claim checks pass; the paragraph’s thesis is still unsupported.
4. Second-sample corroboration. Re-asking the model returns the same citations, felt as confirmation. Agreement sampled twice from the same failure mode is not proof.

OPINION: a literature claim without a source hash is a rumor with a bibliography. Debugging research means treating every citation as a pointer to be dereferenced, not a credential to be admired.

The failure has been measured, with the usual caveat that model behavior changes. Walters and Wilder had GPT-3.5 and GPT-4 write short literature reviews on 42 topics and checked all 636 resulting citations: 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated outright, and among the real citations, 43% (GPT-3.5) and 24% (GPT-4) contained substantive errors (Walters & Wilder, 2023). Those numbers are 2023 models without retrieval; retrieval grounding reduces the problem but does not close it. Liu and colleagues audited four citation-first generative search engines and found only about half of generated statements fully supported by their own cited sources, and about a quarter of citations not supporting the statement they were attached to — with a response’s fluency and its verifiability inversely related (Liu et al., 2023). In a high-stakes domain the residual is starker: Magesh and colleagues’ preregistered audit of commercial retrieval-augmented legal-research tools found hallucination rates of 17% to 33%, much of it misgrounding — misdescribing a real case, citing inapplicable authority — rather than outright fabrication, and the vendors’ “hallucination-free” claims overstated (Magesh et al., 2025). That is exactly why you resolve to the retrieved byte rather than trust the rate or the vendor.

The mental model: Chapter 3’s evidence hygiene applied to prose about the world. The two-column worksheet returns with higher stakes: EVIDENCE (retrieved bytes with hashes and timestamps) vs. EXPLANATION (every sentence with “studies show,” “as proven by,” “following”). Delete the right column; what stands on the left alone is the research.

The method: resolve every reference to a retrieved byte

Freeze the research text, then dereference it claim by claim:

1. Extract claims. Number every factual sentence (C1…Cn), separating factual claims from synthesis/opinion sentences labeled as such.
2. Resolve. For each claim, record the retrieved document (ID + content hash + retrieval timestamp + query that returned it) and quote the supporting byte range. No hash → H1 candidate. Hash present but content contradicts → H2. Content supports atoms but not the combined conclusion → H3.
3. Run the fabricated-reference drill. For each unresolvable citation: search the quoted title/DOI independently (catalog lookup, distinct query path), record the MEASUREMENT (resolves / resolves-to-other / unresolvable), and downgrade the dependent synthesis to UNKNOWN — never to “probably fine.”

    flowchart TD
    N["number every factual claim C1..Cn; label synthesis / opinion separately"] --> L["independent lookup per citation — a query path distinct from generation"]
    L --> R{"citation resolves to a retrieved byte?"}
    R -->|"no matching byte"| H1["H1 fabricated-reference — replace the source or delete the claim"]
    R -->|"resolves"| S{"quoted bytes support the claim's number and direction?"}
    S -->|no| H2["H2 misattributed — correct the claim against the quoted bytes"]
    S -->|yes| OK["claim supported — keep with hash + timestamp"]
    H1 --> SY{"synthesis row: any dependency H1 / H2 / UNKNOWN?"}
    H2 --> SY
    OK --> SY
    SY -->|yes| U["conclusion is UNKNOWN — block it, never 'probably fine'"]
    SY -->|no| SUP["synthesis SUPPORTED only if it does not exceed the combined atoms (H3 check)"]
  

PROVENANCE TABLE (research frozen; bytes only):
claim | citation string | retrieved doc (hash/timestamp) | quoted support bytes | verdict
C1: "X improves recall 12%" | Doe 2023, doi:___ | doc-07 h:44c0, 2026-09-01 | "recall +2.1% (§4)" | H2 misattributed (number contradicts)
C2: "Y is standard for Z" | Smith 2021, doi:___ | — none retrieved — | — | H1 fabricated-reference candidate
C3: "therefore X+Y dominates" | C1 + C2 | (depends on C1, C2) | — | H3 overreach + UNKNOWN until C1/C2 resolve
RULE: delete every row without a hash. What remains is the research; the rest is draft.

OBSERVATION (constructed illustration, not a measured run): of nine factual claims in a generated survey, four resolved to supporting bytes, three contradicted their cited sources on numbers (H2), and two had no retrievable source under independent lookup (H1); the survey’s concluding recommendation depended on all five defective rows. UPDATED BELIEF: H1 supported for two rows, H2 for three, H3 for the conclusion — for this instance only. The recommendation is UNKNOWN, not “partially supported.” MEASUREMENT vs. claim hygiene: counts above are illustration scaffolding showing the table’s shape; no empirical rate about AI citation accuracy is asserted.

Example: dereferencing a three-citation paragraph

Generated paragraph (frozen): “Prior work establishes segment-cached invalidation as standard (Smith 2021), with 12% recall gains reported at scale (Doe 2023), so we adopt it as the proven baseline (synthesis).”

# provenance resolution: every reference dereferenced, contradictions quoted
claims = {
    "C1": ("segment-cached invalidation is standard", "Smith 2021"),
    "C2": ("12% recall gains at scale", "Doe 2023"),
    "C3": ("adopt as proven baseline", "synthesis of C1+C2"),
}
# Step 1: independent lookup per citation (distinct query path from generation's).
# Step 2: record MEASUREMENT per row: resolves-exact / resolves-to-other / unresolvable.
# Step 3: quote supporting bytes or mark absent; never paraphrase-from-memory.
# FORECASTS pre-written: H1 -> unresolvable under independent lookup; H2 -> resolved
# doc contradicts the number/direction; H3 -> atoms resolve, combination exceeds them.
# C3 verdict derives from C1+C2: if either is H1/H2, C3 is UNKNOWN (not "weakened").

In the constructed case: Smith 2021 resolves but describes write-through (not segment-cached) invalidation — H2, quoted bytes contradict the claim. Doe 2023’s DOI resolves to an unrelated paper and no content matching the 12% figure is retrievable — H1 after independent lookup. The synthesis (C3) therefore collapses to UNKNOWN regardless of its eloquence: conclusions built on unresolved rows are unresolved.

No fluency, citation count, venue prestige, or model confidence substitutes for byte linkage. A second generation agreeing with the first is the same failure mode sampled twice.

Research lineage: atomic decomposition is the verified method

The claim-numbered provenance table is FActScore. Min and colleagues formalized long-form factuality checking as: break the generation into atomic facts — one short sentence, one piece of information — and compute the fraction supported by a reliable knowledge source; their human study put ChatGPT biographies at 58% supported, and their automated retrieval-plus-LM estimator matched human judgment within about 2% (Min et al., 2023). This chapter’s step 1 (“number every factual sentence”) and step 2 (“resolve each to a retrieved byte”) are FActScore run by hand, with the synthesis-derivation rule added on top.

Post-hoc research-and-revise is the constructive version. Gao and colleagues’ RARR takes a model’s output, retrieves evidence for each claim, and edits the claims that the evidence does not support while preserving the rest (Gao et al., 2023). The provenance table’s H2 rows (“correct the claim against quoted bytes”) are RARR’s revision step.

In high-stakes domains the base rate is worse. Dahl and colleagues profiled legal hallucination and found that leading models produced incorrect information on a majority of specific legal queries, often with unwarranted confidence (Dahl et al., 2024). Their follow-up (Magesh et al.) extended this to retrieval-augmented commercial legal tools and found grounding cut the rate but not below 17%. The drill’s rule — downgrade unresolved synthesis to UNKNOWN, never “probably fine” — is calibrated for exactly this.

Lab 28: the fabricated-reference drill (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own provenance table.

Setup. Take one AI-generated research passage (literature review, prior-art section, or factual summary) with ≥5 citations. Freeze the text, model version, seed, and any retrieval logs from generation. An independent lookup path (catalog, index, or search distinct from the generation pipeline) is the instrument; everything else is controlled.

Task.

1. Before resolving, write H1/H2/H3 with distinct predicted row patterns: H1: “citation unresolvable under independent lookup (no matching byte)”; H2: “resolves but quoted bytes contradict the claim’s number or direction”; H3: “atoms resolve, paragraph conclusion exceeds their combination.”
2. Build the provenance table: every factual claim numbered, every citation independently looked up, support bytes quoted or absence recorded as MEASUREMENT, synthesis rows derived (any defective dependency → UNKNOWN).
3. Record OBSERVATION (per-row verdicts verbatim) and UPDATED BELIEF per hypothesis. Claims with vendor-asserted sources (“per vendor docs”) are marked changeable-fact rows requiring independent retrieval — vendor attribution is a lead, not a hash.

Success criterion. A completed provenance table with every factual claim resolved-or-marked, independent-lookup measurements recorded, and synthesis rows derived as UNKNOWN where dependencies fail. A plausibility read (“references look legitimate”) is explicitly not completion.

Companion tool: Research Validation Checklist

What it accepts: the frozen research text, the numbered claim list, per-claim citation strings, independent-lookup measurements, and quoted support bytes with document hashes/timestamps. What it performs: it blocks research acceptance while any factual claim lacks a hash-linked support quote, enforces independent lookup for every citation (distinct path from generation), derives synthesis verdicts from dependency rows, and requires repeated lookups where the index is nondeterministic. What it can establish: whether each claim resolves to a retrieved byte and which rows are fabricated, misattributed, or overreaching — for the examined passage only. What it cannot establish: source truth beyond the retrieved bytes (a faithfully quoted but wrong paper stays wrong), completeness of the literature (unseen sources are UNKNOWN, not absent), or future citation reliability. It never treats citation shape, fluency, confidence, agreement, or venue as diagnosis. How its output changes your next action: H1 rows route to source replacement or claim deletion; H2 rows route to claim correction against quoted bytes; H3 routes to synthesis narrowing with explicit hedging; any UNKNOWN row blocks dependent conclusions until resolved.

Paper form, sufficient for this chapter:

Passage hash: ___  Model/seed: ___ / ___  Lookup path: ___ (distinct from generation: Y/N)
Claims (n=___): resolved ___ | contradicted ___ | unresolvable ___ | opinion-labeled ___
Per row: Cn ___ | cite ___ | lookup: exact/other/none | bytes: "___" / none | verdict H1/H2/H3/OK
Synthesis rows: ___ -> UNKNOWN / SUPPORTED (dependencies: ___)
BLOCKED CONCLUSIONS: ___

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Resolve before relying.

Reusable procedure: bytes before belief

1. Freeze the passage — text, hashes, model/seed, generation retrieval logs.
2. Number the claims — factual vs. synthesis/opinion, each labeled.
3. Look up independently — distinct path, per-citation MEASUREMENT recorded.
4. Quote or mark absent — support bytes with hashes, or explicit none.
5. Derive synthesis — any defective dependency makes the conclusion UNKNOWN.

Failure modes

• Plausibility review. Titles scanned, DOIs admired. Fabrication is designed to pass exactly this.
• Existence-as-support. DOI resolves, claim assumed verified. Resolution proves existence; only quoted bytes prove support.
• Paraphrase-from-memory. “The paper basically says…” Quotation or UNKNOWN — memory is not retrieval.
• Synthesis laundering. Faithful atoms, unsupported thesis. Check the combination, not just the components.
• Agreement-as-corroboration. Re-prompting returns the same citations. Same mode, same failure, second sample.
• Vendor-claim import. “Vendor docs state 99.9%” without retrieved bytes. Attribute, mark changeable, verify independently — never file as support. (Measurably: vendor “hallucination-free” claims for retrieval-augmented legal tools were found overstated — Magesh et al.)

Limits, per contract: one table validates one passage under one lookup path and index revision; it does not certify the field, does not prove source correctness, and does not survive index changes without re-resolution. UNKNOWN where lookup paths share the generation pipeline.

References

• William H. Walters and Esther Isabelle Wilder. Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT. Scientific Reports 13, 14045, 2023. https://doi.org/10.1038/s41598-023-41032-5
• Nelson F. Liu, Tianyi Zhang, and Percy Liang. Evaluating Verifiability in Generative Search Engines. Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7001–7025. https://arxiv.org/abs/2304.09848
• Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, and Daniel E. Ho. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies 22(2), 2025, pp. 216–242. https://arxiv.org/abs/2405.20362
• Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FActScore: Fine-Grained Atomic Evaluation of Factual Precision in Long Form Text Generation. Proceedings of EMNLP, 2023, pp. 12076–12100. https://doi.org/10.18653/v1/2023.emnlp-main.741
• Luyu Gao, Zhuyun Dai, Panupong Pasupat, et al. RARR: Researching and Revising What Language Models Say, Using Language Models. Proceedings of ACL, 2023, pp. 16477–16508. https://doi.org/10.18653/v1/2023.acl-long.865
• Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E. Ho. Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis 16(1), 2024, pp. 64–93. https://doi.org/10.1093/jla/laae003

Debugging Checklist

• Passage frozen (text + hash + model/seed + generation logs)?
• Every factual claim numbered (synthesis/opinion labeled separately)?
• Independent lookup per citation (distinct path, MEASUREMENT recorded)?
• Support bytes quoted with hashes (or absence explicitly recorded)?
• H1/H2/H3 FORECASTs pre-written with distinct predicted row patterns?
• Synthesis rows derived (defective dependency → UNKNOWN, never “partial”)?
• No shape, fluency, confidence, agreement, or venue cited as support?

What This Chapter Established

• Citation/provenance verification for AI research: the claim-numbered provenance table with independent lookup, the fabricated-reference drill, and synthesis-derivation rules — demonstrated on constructed passages, no measured runs or accuracy rates claimed.
• Direct tie to Chapter 3: model-generated references are explanations (hypotheses about the literature), never trace evidence; only retrieved bytes with hashes count.
• Lab 28 as a proposed provenance record the reader executes; the Research Validation Checklist contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any assertion about any real paper or any certification of this reader’s literature section. The fabrication rates cited (Walters & Wilder: 55% GPT-3.5 / 18% GPT-4 fabricated in 2023) are bounded to that study and those models — the method exists because the rate is not zero and not knowable in advance.
• Research grounding: the claim-numbered provenance table is FActScore’s atomic-fact decomposition run by hand (Min et al.); the H2 “correct against quoted bytes” step is RARR’s revision (Gao et al.); the UNKNOWN-not-“probably-fine” rule is calibrated for high-stakes domains where the base rate is worse (Dahl et al. on legal hallucination). Retrieval grounding reduces but does not eliminate the problem — only ~half of statements in citation-first generative search are fully supported by their citations (Liu et al.), commercial RAG legal tools still hallucinate 17–33% (Magesh et al.), and misgrounding (H2), not fabrication (H1), is the retrieval-era residual.
• Forward link: research verified — yet the coding agent that consumed it still loops: plan, edit, test, re-plan, same edit, same failure. Document debugging is clean; trajectory debugging is Chapter 29’s jurisdiction.

Next

The sources check out and the agent still spins — re-reading the same files, re-applying the same patch, re-running the same failing test, then declaring victory on a red suite. Provenance is innocent; behavior over time is guilty. Chapter 29, “Debugging Coding Agents,” triages code-agent trajectories: loop detection, test-gating, and the minimal-trajectory repro that separates a stuck agent from a hard problem.