Explore why language models hallucinate, how hallucination can be measured and evaluated, where individual detection methods fail, and how evidence, verification, policy, abstention, and memory gates can be combined to build reliable systems around stochastic models.
A language model can produce an answer that is fluent, confident, detailed, and wrong.
That is the hallucination problem.
But hallucination becomes much more interesting once we stop treating it as a mysterious defect inside the model and start asking engineering questions about it.
What exactly failed?
Can the failure be measured?
How do we know whether a detector actually works?
What happens when a claim is supported semantically but wrong relationally?
When should a model refuse to answer?
How do we distinguish a useful answer from one that is merely safe and generic?
And, most importantly:
How do we build reliable systems around models that cannot themselves be assumed reliable?
This book approaches hallucination from that direction.
It begins with the model, but it does not end there.
The book moves from the causes and forms of hallucination into evidence, measurement, benchmarks, geometric detection, adversarial evaluation, consistency, sensitivity, uncertainty, verification, policy, and memory.
The recurring idea is simple:
Generation may be stochastic. Acceptance does not have to be.
A language model can remain probabilistic.
It can explore.
It can generate alternatives.
It can make mistakes.
The surrounding system can still measure those mistakes, verify important claims, demand evidence, request revision, abstain when necessary, reject unsafe outputs, and prevent failures from becoming persistent state.
That is the larger subject of this book.
A simple view of hallucination looks like this:
graph LR
P[prompt] --> LM[language model] --> A[answer] --> O[correct or hallucinated]
This is useful as a starting point.
It is not enough for building reliable systems.
In practice, an output may fail in several different ways:
unsupported information
incorrect relationships
invented sources
misrepresented evidence
false claims about tools or system state
overconfidence when evidence is insufficient
generic answers that ignore the actual problem
correct information that cannot be verified
bad outputs that become memory
These are not necessarily the same failure.
And they cannot necessarily be detected by the same measurement.
The problem therefore expands:
graph TD
G[generation] --> C[candidate output]
C --> E[what evidence exists?]
E --> I[is the output contained by that evidence?]
I --> R[does it preserve the relationships in the evidence?]
R --> U[does it respond to the actual context?]
U --> S[is there enough information to answer?]
S --> V[can the important claims be attributed and verified?]
V --> D[what should the system do?]
The important shift is from judging a whole answer at once to asking which evidential and policy conditions must hold before the system may accept it.
The answer might be:
ACCEPT
REFINE
RETRIEVE
VERIFY
ABSTAIN
REJECT
That is a much richer problem than binary hallucination classification.
It is also much closer to how reliable AI systems actually need to work.
This book is about building reliable systems around models that cannot themselves be assumed reliable.
Measurement and evaluation come first. Before we can control failure, we need to understand what our measurements actually tell us. The later chapters turn those measurements into abstention, policy, verification, repair, and memory control.
By working through the chapters, you will learn how to:
distinguish hallucination from factual error, uncertainty, contradiction, unverifiability, and weak reasoning;
understand why a language model can produce plausible language without possessing a reliable mechanism for determining whether its own output is justified;
separate truth, evidence, containment, attribution, verifiability, and policy;
structure generated text as claims that can be evaluated against evidence;
understand the strengths and weaknesses of semantic similarity, entailment, confidence, self-consistency, model-based judges, and geometric methods;
measure whether a claim extends beyond the semantic information represented by its evidence;
derive and use Hallucination Energy as a projection-residual grounding signal;
evaluate hallucination detectors using precision, recall, ROC-AUC, calibration, false acceptance rates, false rejection rates, and threshold stability;
recognize misleading evaluation results caused by trivial negatives, dataset leakage, weak evidence, ambiguous labels, or domain mismatch;
construct adversarial examples that expose what a detector actually measures;
understand why containment and truth are different;
distinguish extrinsic hallucination from failures that remain inside the evidence manifold;
measure whether an output changes when the important facts of a problem change;
detect context-insensitive, generic answers through perturbation and sensitivity testing;
reason about abstention as a first-class capability rather than a failure to answer;
evaluate the trade-off between hallucination rate, refusal rate, coverage, and usefulness;
combine multiple measurements into a diagnostic reliability vector rather than forcing every failure into one score;
convert measurements into deterministic policy;
design Accept / Refine / Retrieve / Verify / Abstain / Reject routing;
protect agents and adaptive systems from storing their own hallucinations;
understand why memory turns a temporary model error into a long-term systems problem; and
build AI architectures that do not require blind trust in the language model at their centre.
The objective is not to discover a magical hallucination score.
It is to understand the measurement problem well enough that we know what a score means, where it breaks, and what a system should do with it.
The book develops this argument across fifteen chapters and a closing appendix.
01 When Models Make Things Up
Begin with the familiar hallucination problem.
Understand why fluent generation and factual reliability are
different properties, and why plausible language makes failure
unusually difficult to recognize.
02 Hallucination Is Not One Thing
Build a taxonomy of failure.
Separate unsupported claims, contradictions, fabricated provenance,
incorrect system-state claims, uncertainty failures, and
context-insensitive answers.
03 Evidence, Truth, and Verifiability
Ask what it actually means for an answer to be supported.
Separate truth from evidence, containment from attribution,
and factual correctness from what a policy allows a system
to publish or accept.
04 How Do You Measure a Hallucination?
Introduce the major families of hallucination measurement:
similarity, entailment, retrieval, confidence, consistency,
LLM judges, internal representations, and external verification.
Understand what each method observes and what it cannot observe.
05 Hallucination Energy
Build a geometric grounding metric from first principles.
Represent evidence as a semantic subspace, project the claim
into that subspace, and measure the residual that lies outside it.
Turn grounding into a measurable scalar.
06 How to Evaluate a Hallucination Detector
Move from measurement to experimental discipline.
Work through ROC-AUC, precision, recall, calibration,
false acceptance, false rejection, threshold selection,
cross-validation, confidence intervals, and domain transfer.
07 Breaking the Detector
Attack our own measurement.
Construct hard negatives, semantic-overlap adversaries,
relational inversions, and domain shifts.
Learn why an impressive benchmark score tells us little
unless we understand how the benchmark can fail.
08 Containment Is Not Truth
Examine the boundary revealed by adversarial evaluation.
A false claim can remain inside the semantic span of its evidence.
Projection can measure containment without detecting
polarity reversal, relation inversion, or compositional error.
09 Beyond Hallucination: Consistency and Sensitivity
Expand the model from one scalar to several independent signals.
Ask whether an output preserves internal structure
and whether it responds to the specifics of its input.
10 The Safe but Useless Model
Explore context-insensitive generation.
Perturb important properties of a problem and test whether
the model's recommendations actually change.
Detect generic output that is grounded, coherent,
and nevertheless fails to reason about the problem.
11 Knowing When Not to Answer
Add epistemic adequacy and abstention.
Ask whether the available evidence is sufficient to justify
an answer at all.
Explore the trade-off between hallucination,
refusal, coverage, and usefulness.
12 From Measurements to Policy
Convert typed measurements into action.
Replace a single hallucination score with a diagnostic vector
and route outputs through explicit policy:
commitment = PERMIT | HOLD | DENY
next_action = NONE | REFINE | RETRIEVE | VERIFY | ASK | REVIEW
13 Verification, Repair, and Rejection
Build the control loop around generation.
Let the model propose an answer, inspect the failures,
attempt bounded repair, verify again, and expose the result
only when the acceptance policy permits it.
14 The Memory Contamination Problem
Move from individual answers to persistent systems.
Show why a hallucination that enters memory is more dangerous
than one bad response: it can become evidence for future reasoning.
Introduce verification before commit.
15 Building Systems That Distrust Their Models
Assemble the complete architecture.
Combine evidence, containment, structural checks,
sensitivity, abstention, attribution, verification,
policy routing, and memory gates.
End with the central systems lesson:
reliability belongs to the whole system,
not to the language model alone.
16 Appendix: The Evidence Ledger
Every empirical claim in one table:
what was measured, on which run, with what
provenance, and what the number does not establish.
The book keeping its own score.
The order matters.
Before discussing detectors, we need to decide what counts as hallucination.
Before measuring grounding, we need to define evidence.
Before trusting a metric, we need to understand evaluation.
Before proposing a multi-axis system, we need to break the single-axis detector.
Before discussing policy, we need measurements that policy can act upon.
And before allowing AI systems to learn from themselves, we need a boundary around what is allowed to become memory.
The progression therefore looks like this:
graph LR
H[hallucination] --> T[failure taxonomy]
T --> E[evidence]
E --> M[measurement]
M --> HE[Hallucination Energy]
HE --> EV[evaluation]
EV --> AT[adversarial testing]
AT --> MB[measurement boundaries]
MB --> MD[multi-axis diagnosis]
MD --> AB[abstention]
AB --> PL[policy]
PL --> VE[verification]
VE --> MP[memory protection]
MP --> RS[reliable system]
The book deliberately moves from naming hallucination to governing the conditions under which generated claims may be accepted, repaired, remembered, or rejected.
A common response to hallucination is to ask how we can make the language model stop hallucinating.
That is an important research problem.
It is not the only engineering option.
Another approach is to accept a more uncomfortable premise:
The generator may remain imperfect.
If we accept that premise, the problem changes.
We no longer require every guarantee to come from the model.
Instead, we construct layers around it.
graph TD
subgraph Stochastic
LLM[LLM]
end
LLM --> C[candidate output]
C --> TM[typed measurements / verification]
subgraph Measurements
TM --> Co[containment]
TM --> Cs[consistency]
TM --> Se[sensitivity]
end
Co --> EC[epistemic check]
Cs --> EC
Se --> EC
EC --> AT[attribution]
AT --> VE[verification]
VE --> PO[policy]
PO --> Out{PERMIT / HOLD / DENY}
Out --> RC[recovery route]
The language model is still useful.
Its stochasticity still provides variation, synthesis, compression, and exploration.
But generation and acceptance are separated.
This distinction is one of the recurring ideas throughout the book:
The model proposes. The system decides what survives.
One of the book’s most important distinctions is between a model’s confidence and the evidence supporting its answer.
A model can be highly confident and wrong.
It can also produce a correct answer for reasons we cannot verify.
High-trust systems need more than plausibility.
They need some combination of:
evidence
provenance
containment
attribution
verification
policy compliance
This changes the question from:
Does the model believe this answer?
to:
What supports this answer?
That is a much stronger foundation for measurement.
Several concepts that are often compressed into the word grounding need to be separated.
Consider the following questions:
Is the claim true?
Is information capable of supporting the claim
present in the evidence?
Can the claim be traced to a specific piece
of evidence?
Does that evidence actually entail the claim?
Is the source acceptable under the active policy?
Is the system allowed to expose the claim?
These are different questions.
A claim can be true but unsupported by the supplied evidence.
It can be semantically contained by a document without being explicitly attributable to a particular sentence.
It can be supported by a source that a strict policy does not permit.
It can be plausible while still requiring the system to abstain.
The book therefore avoids treating grounding as a single undifferentiated property.
Instead, we progressively strengthen the guarantee:
semantic containment
↓
specific attribution
↓
claim verification
↓
source verification
↓
policy acceptance
Each layer tells us something new.
The first major measurement developed in the book is Hallucination Energy.
Suppose we have:
claim: c
evidence: e₁, e₂, ... eₙ
The evidence embeddings define a semantic subspace.
We project the claim into that subspace.
graph TD
E[evidence subspace] --> P[projection of claim]
C[claim] --> P
C --> R[residual]
For a unit-normalized claim and an orthonormal evidence basis, Hallucination Energy is the squared residual:
|\mathbf c-\mathbf B\mathbf B^T\mathbf c|_2^2 $$
Intuitively:
low energy
claim lies largely inside
the evidence span
high energy
significant semantic content
lies outside the evidence span
This gives us something extremely useful:
a model-independent, thresholdable measurement of containment
But it is important to understand the word containment.
Hallucination Energy does not become a truth oracle simply because it produces a number.
That distinction drives the next stage of the book.
A measurement is only useful if we understand how it fails.
Suppose a hallucination detector achieves:
AUC = 0.95
That may be excellent.
Or the experiment may be trivial.
If positive claims come from the correct article and negative claims come from unrelated articles, the detector may simply have learned:
unrelated text is unrelated.
That is not the same as detecting hallucination.
The evaluation therefore matters as much as the metric.
Throughout the book we repeatedly ask:
How were the negatives constructed?
How much semantic overlap exists?
Are labels actually measuring hallucination?
Is evidence explicit?
Are the examples atomic or long-form?
Does the detector survive paraphrase?
Does it survive relation inversion?
Does it survive hard-negative mining?
Does the threshold transfer between domains?
Does performance collapse when examples become adversarial?
What happens to refusal and coverage?
A hallucination detector should not only be demonstrated.
It should be pressured until its boundary becomes visible.
The most important result in the Hallucination Energy experiments is not that the metric works.
It is where it stops working.
Consider two claims:
Evidence:
Alice founded Acme before Bob became CEO.
Claim A:
Alice founded Acme before Bob became CEO.
Claim B:
Bob founded Acme before Alice became CEO.
Both statements may occupy almost the same semantic region.
They contain the same entities.
They contain similar vocabulary.
They discuss the same relationships.
One is supported.
The other may reverse the structure.
A containment detector can therefore encounter:
true claim → low residual
false claim → low residual
The detector has not malfunctioned.
It is doing exactly what it was designed to do.
It measures:
Is the claim inside the evidence space?
It does not necessarily measure:
Are all relationships inside the claim correct?
That gives us a general principle:
Every detector has a boundary determined by the property it measures.
This principle applies far beyond Hallucination Energy.
Similarity has a boundary.
Entailment models have a boundary.
Self-consistency has a boundary.
Confidence has a boundary.
LLM judges have a boundary.
Internal activation detectors have a boundary.
The correct response is not to hide those boundaries.
It is to map them.
Once a single scalar is no longer enough, the architecture changes.
Instead of asking:
hallucination = 0.63
we can produce something closer to:
containment PASS
structural consistency PASS
context sensitivity FAIL
evidence sufficiency PASS
attribution PARTIAL
The output becomes diagnostic.
A useful conceptual representation is:
(H,C,S,E,A,\ldots) $$
where the dimensions might represent:
H containment / hallucination energy
C consistency / structural fidelity
S sensitivity / context responsiveness
E epistemic adequacy / answerability
A attribution / evidence traceability
The exact vector can change by application.
That is the point.
A medical system may care deeply about attribution and abstention.
A creative writing system may tolerate semantic expansion.
A coding agent may care about executable verification.
A research system may require provenance for every important claim.
Policy determines which dimensions matter and where their thresholds sit.
A model does not need to fabricate information to fail.
Consider two very different situations:
A struggling startup
with six months of runway
A highly profitable market leader
in a rapidly expanding industry
Now imagine the model recommends the same things in both situations:
focus on innovation
improve efficiency
listen to customers
use data to guide decisions
Nothing is obviously false.
The output might be:
grounded ✓
coherent ✓
safe ✓
useful ?
The problem is that the answer barely responds to the input.
This failure is difficult to detect from a single output.
We therefore change the experiment.
Instead of evaluating only the answer, we modify the problem.
base problem
↓
polarity change
scale change
constraint change
domain change
↓
regenerate
↓
compare outputs
If important changes in the problem produce almost no meaningful change in the answer, the output surface is too flat.
This introduces sensitivity as another measurable property.
A reasoning system should respond when its problem changes.
There is another obvious escape from hallucination:
Question
↓
"I don't know."
A model that refuses everything has an excellent hallucination rate.
It is also useless.
This means hallucination cannot be evaluated independently of coverage.
The system needs to distinguish:
correct answer
incorrect answer
appropriate abstention
unnecessary refusal
That produces another fundamental trade-off:
answer more
↓
greater coverage
↓
potentially more hallucination
refuse more
↓
less hallucination
↓
potentially less usefulness
A good system should not merely minimize hallucination.
It should abstain intelligently.
That requires measuring whether the available evidence is sufficient for the requested conclusion.
The question is no longer:
Can the model produce an answer?
It becomes:
Does the system have enough justified information to permit one?
Measurements alone do not make a system reliable.
They produce information.
Something still has to decide what to do.
That is the role of policy.
A policy might specify:
if containment fails:
HOLD / RETRIEVE or DENY
if consistency fails:
HOLD / VERIFY or DENY
if sensitivity fails:
HOLD / REFINE or HOLD / REVIEW
if evidence is missing:
HOLD / RETRIEVE
if the claim is high-stakes:
HOLD / VERIFY
if the question is not answerable:
HOLD / ASK or DENY
otherwise:
PERMIT / NONE
The exact rules differ by system.
But the architectural principle remains constant:
Measurement describes the candidate. Policy determines its fate.
This is where probabilistic generation can meet typed measurements, verification, and explicit policy.
Once measurement and policy are combined, hallucination management becomes a runtime process.
graph TD
G[generate] --> E[evaluate]
E --> D[diagnose]
D --> R[route]
R --> A[accept]
R --> RF[refine]
R --> RV[retrieve]
R --> RJ[reject]
RF --> G
RV --> G
A failure does not necessarily end the process.
Some failures are repairable.
A model may:
Reliable systems need different responses for different failure modes.
That is why the diagnostic vector matters.
A hallucinated chatbot response may disappear after one interaction.
A hallucination stored in persistent memory is different.
It can become part of the system’s future context.
The feedback loop becomes:
graph TD
H[hallucinated output] --> S[stored as memory]
S --> R[retrieved later]
R --> T[treated as evidence]
T --> NG[influences new generation]
NG --> NO[new output]
NO --> S
A temporary error has become persistent state.
For agents, RAG systems, long-running assistants, and self-improving systems, this changes the importance of verification.
The relevant boundary is no longer only:
Can this answer be shown to the user?
It is also:
Can this information become memory?
The runtime therefore changes from:
generate
↓
store
to:
generate
↓
verify
↓
repair / reject
↓
commit
The model can still make mistakes.
The system does not have to remember them.
This leads to the broadest argument in the book.
It is tempting to speak of a reliable model.
But many guarantees that matter in real systems do not have to come from the model itself.
Consider the layers surrounding generation:
graph TD
M[MODEL] --> CO[candidate output]
CO --> G[grounding measures]
G --> SC[structural checks]
SC --> ST[sensitivity testing]
ST --> EA[epistemic adequacy]
EA --> AT[attribution]
AT --> VE[verification]
VE --> PO[policy]
PO --> OUT[PERMIT / HOLD / DENY + next action]
OUT --> MG[memory gate]
Some of these layers may use another model.
Some may use deterministic code.
Some may use databases, search engines, theorem provers, compilers, test suites, schemas, external APIs, or human review.
The important idea is separation of responsibility.
The generator generates.
Evidence systems provide evidence.
Measurements measure.
Verifiers verify.
Policy decides.
Memory gates control persistence.
The model no longer has to be both the source of an assertion and the sole authority deciding whether that assertion should be trusted.
The goal is not that you memorize one hallucination benchmark or one detection technique.
The goal is that you can look at an AI system and ask better questions.
You should be able to ask:
What exactly counts as hallucination here?
What evidence is available?
What does this detector actually measure?
Is it measuring truth, containment, attribution,
consistency, or something else?
How were the negative examples constructed?
Could the benchmark be artificially easy?
What is the false acceptance rate?
What is the false rejection rate?
How was the threshold calibrated?
Does it transfer to another domain?
What adversarial examples break it?
Can a false claim remain inside the detector's
accepted region?
Does the model respond when important context changes?
Does it know when evidence is insufficient?
What is the refusal rate?
What is the coverage?
Who verifies the important claims?
What happens when verification fails?
Can the system repair the output?
When should it reject it?
Can a failed output become memory?
What prevents a hallucination from becoming
future evidence?
Where is the deterministic boundary
around the stochastic generator?
Once you can ask those questions, hallucination becomes much less mysterious.
It becomes an engineering problem.
This book is for people who want to understand hallucination beyond the familiar observation that language models sometimes make things up.
It is especially useful for:
developers building applications around large language models;
engineers building RAG and retrieval systems;
researchers studying factuality, grounding, evaluation, and reliability;
developers building agents, persistent assistants, and self-improving systems;
people designing AI evaluation infrastructure;
people responsible for deploying models in environments where unsupported claims matter;
anyone interested in how probabilistic generation can coexist with deterministic software guarantees; and
anyone who wants to understand what hallucination detectors actually measure rather than treating benchmark scores as answers in themselves.
Some chapters are mathematical.
Some are experimental.
Some are architectural.
Some deliberately document failed hypotheses and measurement boundaries.
Together they tell one story:
We can understand unreliable generation well enough to build systems that do not have to trust it blindly.
This book does not claim that hallucination has been solved, nor that Hallucination Energy is a universal truth detector. Embedding geometry will not capture every hallucination, and another language model cannot be relied upon to judge every answer correctly. Deterministic policy, meanwhile, should not be confused with a deterministic generator.
Nor does the argument depend on every application using the same thresholds, signals, or evaluation layers. Different systems fail in different ways, under different costs and different tolerances for error.
Instead, the book develops a methodology:
define the failure
measure it
evaluate the measurement
attack the measurement
discover its boundary
add another signal when necessary
calibrate policy
verify important claims
control persistence
The objective is not perfect generation.
It is controlled acceptance.
A model may continue to produce uncertain, unsupported, or incorrect statements. The engineering problem is to decide which outputs are allowed to matter: which claims are accepted, which are verified, which are rejected, and which are prevented from becoming persistent state.
That changes the problem. We no longer need a detector that recognizes every hallucination in every possible context. We need a system that makes consequential acceptance increasingly difficult for unsupported claims.
No individual layer is sufficient. Geometry can fail. Judges can fail. Retrieval can fail. Verification can fail. Policies can be miscalibrated. But these failures need not occur in the same way or at the same time.
The practical goal is therefore not to eliminate uncertainty from generation. It is to build enough independent structure around generation that uncertainty does not silently become fact.
That is the central claim of this book:
hallucination is not only a generation problem. It is an acceptance problem.
By the end of Hallucination From First Principles, you should see hallucination not as one mysterious defect, but as a collection of measurable failure modes.
You should know why evidence matters, where containment differs from truth, and why a detector that performs well on one benchmark may collapse under adversarial testing. Factuality is only part of the picture: consistency, sensitivity, abstention, attribution, and coverage also shape whether a system can be trusted.
More importantly, you should be able to evaluate a hallucination detector rather than accept its headline score at face value.
From there, the problem becomes architectural. Model output can pass through explicit measurement, verification, and policy before it is accepted—or allowed to become memory.
The destination is not a model that never fails.
It is something more practical:
A system that expects its model to fail, knows how to measure the failure, and has explicit rules for deciding what survives.
That is the larger subject of the book.
It begins with hallucination.
It ends with reliability.
Start from the first-principles hallucination problem: why a language model can remain fluent after evidential support runs out, why guessing and abstention create a real engineering trade-off, and why reliable systems must separate generation from acceptance.
Read chapter →Build a multidimensional engineering taxonomy for LLM hallucination by separating reference, failure relation, measurement unit, information path, and propagation, then connect each failure to the detector that can actually observe it.
Read chapter →Separate resolved claims, truth, evidence, support, attribution, provenance, source reliability, sufficiency, temporal validity, verifiability, verification, and policy so that hallucination measurements have a precise reference and decisions have an explicit basis.
Read chapter →Treat hallucination detectors as explicit sensors with target properties, references, granularity, proxy gaps, costs, and blind spots; then map lexical, semantic, entailment, retrieval, uncertainty, judge, internal-state, counterfactual, programmatic, runtime, provenance, and geometric signals into a composable measurement architecture.
Read chapter →Derive Hallucination Energy from first principles as a squared projection-residual measurement of embedding-subspace containment, connect the mathematics to the Certum implementation, and test both the practical signal and structural boundary of the proxy.
Read chapter →Build a rigorous evaluation discipline for hallucination detectors: separate measurement quality from decision quality, rank scores from operating points, calibrate explicit risk budgets, quantify uncertainty, prevent leakage, construct hard negatives, test transfer, and evaluate the verifier pipeline rather than trusting one AUC number.
Read chapter →Turn hallucination evaluation into adversarial measurement science: define explicit threat models, separate baseline errors from attack-induced failures, organize attacks across content, evidence, configuration and verifier pipelines, enforce oracle validity, measure adversarial degradation under search pressure, and preserve discovered boundaries as held-out and regression tests.
Read chapter →Explain why embedding-subspace containment is neither support nor truth: trace information loss from language and evidence through representation, basis construction, projection, scalar reduction, and policy; formalize measurement aliasing; separate semantic material from propositional structure; and derive the need for structural and multi-axis verification.
Read chapter →Move beyond a single hallucination score by treating reliability as transformation fidelity: test whether stochastic response distributions remain stable under changes that should not matter, move appropriately under changes that should matter, validate perturbation oracles, and combine containment, constraint fidelity, and counterfactual responsiveness in a typed diagnostic reliability record.
Read chapter →Examine context-insensitive convergence: outputs that look safe under static checks yet barely depend on the problem. Define default response basins as behavioral objects, separate marginal convergence from scenario correctness, formalize context ablation, counterfactual inversion, decisive-fact and reversal-condition tests, and build an auditable anti-genericity evaluation protocol.
Read chapter →Treat abstention as a measurable reliability capability rather than a failure: distinguish evidence insufficiency from model uncertainty and policy refusal, formalize selective prediction and risk-coverage trade-offs, route recoverable uncertainty to retrieval or clarification, preserve partial answers, and evaluate whether a system knows when it has enough evidence to answer.
Read chapter →Turn typed reliability measurements into executable system behavior. Separate diagnostic interpretation from authorization, model commitment and recovery as different policy dimensions, encode hard constraints and policy floors, preserve unknown and intentionally skipped states, bound recovery loops, version and replay policy decisions, and enforce decisions outside the generator.
Read chapter →Turn policy routes into bounded recovery loops. Separate verification from self-critique, treat every repair as a new candidate state, localize failures before editing, track surface and dependency blast radius, preserve cohesion and repair provenance, re-measure affected dependencies, detect repair-induced regressions and cycles, and terminate with omission, abstention, review, or rejection when recovery no longer earns trust.
Read chapter →Treat persistent AI memory as a governed future-influence channel. Separate origin, verification, temporal validity, lifecycle and use capabilities; gate writes and reads; preserve claim-level lineage; prevent summarization and retrieval from laundering unverified claims; and invalidate contaminated state before it becomes future evidence or action.
Read chapter →Assemble the book's proposal, observation, measurement, policy, enforcement, recovery, and memory components into one closed reference architecture. Mediate assertion, action, and persistence through separate gates, preserve replayable lineage, scale controls with consequence, monitor the control plane itself, and earn bounded reliability without pretending the model is infallible.
Read chapter →Every empirical claim in the book in one place: what was measured, on which run, with what provenance, and what the number does not establish. The book applies its own discipline to itself.
Read chapter →