How Do You Measure a Hallucination?

The first three chapters deliberately avoided building a detector.

Before designing one, we needed to define exactly what it would be expected to detect.

Chapter 1 established that fluent generation can continue after evidential support has weakened or disappeared.

Chapter 2 showed that hallucination covers several different failure relationships.

Chapter 3 then separated the objects a reliability system must not collapse:

truth
≠
evidence
≠
support
≠
attribution
≠
provenance
≠
verification
≠
policy acceptance

Now we can finally ask the engineering question:

What can software actually measure?

Suppose we have a resolved claim:

Company A acquired Company B in 2024.

A reliability system might inspect:

word overlap

embedding similarity

entailment probability

retrieval relevance

source metadata

agreement across independent sources

model confidence

variation across repeated generations

semantic entropy

another model's judgment

hidden-state activations

counterfactual response sensitivity

an executable database query

execution traces

geometric distance from an evidence representation

All of those can be useful.

They are not measurements of the same thing.

That gives us the central rule of this chapter:

A hallucination measurement is a sensor, not a verdict.

Every sensor observes some property of a candidate through some observation channel.

Every sensor throws information away.

The job is therefore not to find the metric with the most impressive name.

It is to specify what the metric can observe, what latent property we hope it approximates, and where those two can come apart.

That separation is the beginning of measurement.

1. Every detector lives across a proxy gap

What we care about is usually latent.

We want to know things such as:

Is this claim true?

Is it supported by the evidence?

Does the model know enough to answer?

Did the claimed tool action really happen?

Does the answer respond to the decisive facts of the problem?

What software actually receives are observable signals.

For example:

The dangerous move is:

proxy
    ↓
correlates with target on one evaluation
    ↓
proxy gets renamed as target

That is how we end up with claims such as:

high similarity = grounded

high confidence = true

high consistency = correct

high retrieval score = evidence

low semantic entropy = factual

None of those implications holds in general.

We will call the distance between the property we care about and the signal we can observe the proxy gap.

A detector is useful when its proxy tracks the target sufficiently well under the conditions we care about.

A detector fails when the target and proxy decouple.

This gives us a scientific way to think about hallucination measurement:

Define the proxy. State why it should track the target. Then construct cases designed to make them diverge.

That is the method we will use for the rest of the book.

2. Measurement begins with target, reference, and granularity

A score means nothing until we specify the measurement contract that produced it.

For sensor \(i\), write:

c   = candidate claim or object under evaluation
r_i = observation/reference available to sensor i
g_i = measurement granularity
s_i = observed score or state

The reference might be:

no external reference

the model's token distribution

other samples from the same model

a supplied document

a retrieved corpus

a provenance graph

a knowledge base

a runtime trace

the open web

The granularity might be:

token

claim

claim–passage pair

claim triplet

sentence

document

response pair

source set

trajectory

This dimension matters.

A sentence-level entailment model can contain useful signal and still fail if it is asked to judge an entire document as one undifferentiated unit.

A claim detector can be excellent while the upstream claim extractor silently merges two propositions or drops a numerical qualifier.

So the unit is not a formatting detail.

Granularity is part of the measurement.

A resolved claim in this chapter means that the system has already tried to identify the proposition being judged: entities, relation, polarity, scope, time, quantity, modality, and relevant qualifiers. That resolution can itself be wrong, which means extraction and normalization are part of the measurement pipeline rather than infallible preprocessing.

3. A score is not a decision

Suppose a detector returns:

0.82

What does that mean?

Without a contract, almost nothing.

It could mean:

82% lexical overlap

cosine similarity of 0.82

82% entailment probability

82% estimated correctness probability

82% agreement across samples

82% of extracted atomic claims supported

Even when the metric is known, the number is still not an acceptance decision.

The architecture should remain:

    graph LR
    C[candidate + observation] --> M[measurement]
    M --> S[score / typed state]
    S --> CAL[calibration]
    CAL --> POL[policy + intended action]
    POL --> D[decision]
  

Calibration turns detector output into policy input by making scores comparable to an explicit decision threshold.

Chapter 6 will handle calibration formally. For now, consider the difference between these two designs:

# Unjustified
if score > 0.5:
    reject()

and:

threshold = calibrated_threshold(
    detector=detector,
    domain=current_domain,
    target_false_accept_rate=0.05,
)

decision = policy.route(
    measurement=score,
    threshold=threshold,
    intended_action="publish",
)

The second version is still simplified, but at least it exposes the missing questions.

A continuous score has to earn its operational meaning through evaluation.

And two additional ideas will matter later:

discrimination
    How well does the score rank risky and safe examples?

calibration
    Does a stated probability correspond to observed frequencies?

A detector can be good at one and poor at the other.

We will measure both later.

4. Start from the Claim Verification Object, not from a favorite detector

Chapter 3 built an evidence-bearing candidate rather than passing raw prose through the system.

That object already tells us which measurements are needed.

Conceptually:

This is a much better starting question than:

Which hallucination library should I install?

The CVO identifies an unresolved field.

The unresolved field determines the target property.

The target property determines which class of sensor can possibly help.

That gives us a practical selection rule:

Choose the cheapest sensor that observes the missing property with enough reliability for the intended action.

Not every claim needs every detector.

A runtime fact that can be checked with an exact trace assertion should not be sent through five probabilistic judges first.

A historical claim that exists only in external records cannot be solved by inspecting a local execution trace.

The reference chooses the sensor.

5. Primitive measurements compose into verification pipelines

Several methods are often presented as distinct hallucination detectors even though they are compositions of a smaller set of operations.

Useful primitives include:

D = decompose response into claims

R = retrieve candidate evidence

A = attribute claim to evidence

S = estimate claim–evidence relation

P = inspect provenance / freshness / independence

U = estimate model uncertainty

C = compare repeated or counterfactual outputs

V = verify against authoritative state

Then familiar systems become compositions.

For example:

embedding retrieval
    = R

NLI support checker
    = S_nli

FActScore-style factuality
    = D → R → S → aggregate

LLM judge with supplied evidence
    = S_llm

LLM judge with web access
    = R → A → S_llm

runtime validation
    = V

This matters because composite detectors fail compositionally.

A reported factuality score might be wrong because:

claim extraction failed

or

retrieval missed the evidence

or

attribution linked the wrong passage

or

support classification failed

or

aggregation hid one critical error

We will call this measurement-chain error.

A false alarm from a composite detector does not automatically prove the generator behaved correctly.

It may prove that the verifier failed.

Likewise, a passing score can hide a broken upstream stage.

A production reliability system therefore needs stage-level telemetry rather than only a final factuality number.

6. Authoritative checks: use software when software already knows

Before reaching for semantic measurements, check whether the surrounding system already possesses the authoritative state.

Suppose the model says:

I ran the test suite and all 214 tests passed.

If the runtime has a trustworthy execution trace, ask:

Was the test tool called?

Which command executed?

What exit code returned?

How many tests were reported?

Likewise:

model says file exists
    → filesystem state

model says database returned 143 rows
    → query-result object

model says payment completed
    → transaction state

model says attachment was inspected
    → input registry + trace

These measurements can be exact because the relevant reference is explicit system state.

The priority rule is:

When the system already possesses authoritative state, compare against that state before reaching for probabilistic semantic detection.

There is one assumption underneath the rule:

The instrumentation itself must be complete, correctly associated with the run, current, and trustworthy.

A missing trace event proves little if the tracing system drops events.

Deterministic verification is conditional on reliable instrumentation.

That is still an enormous advantage over semantic guesswork.

7. Programmatic verification: compile claims into checks when possible

Some claims are not runtime observations but can still be reduced to executable checks.

For example:

Claim:
Revenue in 2024 was $5.2 billion.

may become:

SELECT revenue_usd
FROM annual_financials
WHERE company_id = :company
  AND fiscal_year = 2024;

A mathematical claim may become a Python calculation.

A code claim may become a unit test.

A repository claim may become a file or AST query.

A structured event claim may become a database lookup.

Conceptually:

    graph LR
    NLC[natural-language claim] --> CN[claim normalization]
    CN --> EQ[executable query / assertion]
    EQ --> AR[authoritative result]
    AR --> COMP[comparison]
  

This family sits between free-form semantic checking and direct runtime assertions.

The main risk moves upstream:

Did we compile the natural-language claim into the right program?

If the query misrepresents the claim, exact execution only gives us an exact answer to the wrong question.

Programmatic verification is powerful precisely because it makes that translation inspectable.

8. Proximity signals: cheap, useful, and non-directional

The simplest reference-relative measurements ask whether claim and evidence resemble one another.

Lexical overlap

Token overlap, n-gram overlap, edit distance, and ROUGE-style metrics are:

cheap
fast
deterministic
interpretable

But consider:

Evidence:
The study did not find a statistically significant reduction in mortality.

Claim:
The study found a statistically significant reduction in mortality.

Almost every important token is shared.

The proposition has flipped.

Or:

Evidence:
Company A acquired Company B.

Claim:
Company B acquired Company A.

As a bag of words, the material is effectively identical.

The role assignment is reversed.

Lexical overlap therefore measures:

How much surface material is shared?

It does not reliably measure polarity, role, causality, temporal order, or evidential strength.

Embedding similarity

Embeddings replace surface overlap with semantic proximity.

Let:

But semantic proximity is still not directional support.

A reversed acquisition, a negated finding, or an inflated causal statement can remain very close to the source in embedding space.

The proxy gap is now:

semantic proximity
        ≠
evidential licensing

Pairwise cosine also compares one claim vector with one evidence vector.

Real evidence is usually a set:

That geometric move motivates Chapter 5.

It does not eliminate the relation-inversion problem.

A wrong proposition can remain semantically in-span.

9. NLI and entailment: make the direction explicit

Natural-language inference gives us a more suitable abstraction for directional claim–evidence comparison.

Given:

premise    = evidence
hypothesis = claim

an NLI model may output:

ENTAILMENT
CONTRADICTION
NEUTRAL

Now:

Evidence:
The treatment did not reduce mortality.

Claim:
The treatment reduced mortality.

as contradiction rather than high similarity.

SummaC demonstrated another important point: earlier NLI-based factual-consistency systems had suffered from granularity mismatch. Sentence-level NLI models were being applied too coarsely to document-level problems. Segmenting and aggregating pairwise relations recovered useful signal.[1]

This means:

The quality of claim extraction and evidence segmentation upper-bounds the quality of downstream NLI measurement.

NLI is still a trained classifier.

It can inherit annotation artifacts, lexical shortcuts, negation heuristics, domain shift, and prior biases from its training data.

It may also struggle with:

multi-hop evidence

specialized domains

numeric ranges

temporal ordering

long contexts

scope and modality

implicit assumptions

Most importantly, evidential support is broader than textual entailment.

Evidence can increase the rational plausibility of a claim without logically entailing it.

Three randomized trials may support:

The intervention probably reduces hospitalization.

without making that statement a theorem logically entailed by one sentence.

So NLI is best understood as an operational sensor for a particular class of directional relation.

It is not a complete theory of evidence.

10. Retrieval is evidence discovery, not evidence judgment

A retrieval score answers a search question:

Which evidence candidates should we inspect?

It does not answer:

What relationship do those candidates have to the claim?

A passage returned with relevance score 0.91 may:

support the claim

refute the claim

discuss the topic without deciding it

be stale

be circular

come from a weak source

This is why retrieval should remain a primitive rather than being renamed factuality.

A stronger pipeline looks like:

    graph LR
    C[claim] --> RET[retrieve]
    RET --> AT[attribute]
    AT --> AS[assess support / refutation]
    AS --> PR[inspect provenance / freshness]
    PR --> AG[aggregate evidence state]
  

Retrieval only becomes useful for factuality after the retrieved material is linked to the claim, checked for support, and filtered by provenance before aggregation.

Retrieval-backed factuality

FActScore is a clear example of composition. It decomposes long-form output into atomic factual units and measures the fraction supported by a knowledge source, with an automated retrieval-plus-model estimator for large-scale use.[2]

Here, atomic means approximately one independently checkable proposition rather than an entire paragraph containing several relations.

Conceptually:

    graph LR
    LA[long answer] --> AC[atomic claims]
    AC --> RET2[retrieve per claim]
    RET2 --> JUD[judge support]
    JUD --> AGG[aggregate factual precision]
  

The architecture is powerful.

Its failure surface is equally important:

D: claim extraction error
R: retrieval miss
A: wrong attribution
S: support-classification error
P: stale or circular source
G: aggregation hides a critical minority error

A low score may mean the generator hallucinated.

It may also mean the measurement pipeline failed to find or recognize real support.

That ambiguity must remain visible during evaluation.

11. Source and provenance sensors: measure the evidence path itself

Chapter 3 separated support from source fitness.

That distinction needs its own measurements.

Many useful source checks are ordinary metadata operations:

publication date

version / revision

source type

retraction status

author / organization

chain of custody

independence from claim origin

content hash

publication order

These can answer questions such as:

Is this source stale for a current-state claim?

Was this paper retracted?

Are three apparent sources actually copies of one origin?

Did the source exist before the claim it is supposedly verifying?

Does policy require a primary source here?

Circularity is especially important:

AI generates claim X
      ↓
website copies X
      ↓
search retrieves website
      ↓
AI cites website as evidence for X

A semantic support detector may see exact textual agreement.

A provenance graph may reveal that the evidence path has no independent root.

Again, different sensor, different property.

Cross-source conflict

Evidence can also disagree with itself.

For claim \(c\), a system can preserve counts or weights such as:

sources supporting c
sources refuting c
sources neutral / insufficient
sources rejected for provenance

Mixed high-quality evidence is not the same state as no evidence.

The correct action may be:

SURFACE_CONFLICT

rather than:

ACCEPT

or:

HALLUCINATION

Source-set coherence is therefore another measurable property.

12. Self-consistency and semantic entropy: from string stability to meaning stability

When external evidence is absent or expensive, we can use the model’s own stochastic behavior as an observation channel.

Self-consistency

SelfCheckGPT samples multiple responses from a black-box model and looks for divergence or contradiction. Its core intuition is that a model with stable knowledge may produce mutually compatible samples, while confabulation may produce inconsistent ones.[3]

Conceptually:

    graph LR
    P[prompt] --> A1[answer_1]
    P --> A2[answer_2]
    P --> A3[answer_3]
    P --> A4[answer_4]
    A1 --> CM[consistency measurement]
    A2 --> CM
    A3 --> CM
    A4 --> CM
  

Answer consistency is a useful signal only after it is separated into surface stability, semantic stability, and calibration rather than treated as truth by repetition.

The strongest interpretation is:

How stable is the model’s answer under resampling?

Not:

Is the answer true?

For example:

sample 1 → Company B acquired Company A
sample 2 → Company B acquired Company A
sample 3 → Company B acquired Company A
sample 4 → Company B acquired Company A

If the real event ran in the opposite direction:

self-consistency = high
factuality        = low

The temperature trap

Repeated-sample methods need meaningful stochastic variation.

If decoding is fully deterministic, repeated generations collapse to the same path and the method loses the variation it was designed to inspect.

SelfCheckGPT is explicitly sampling-based.[3] Farquhar and colleagues likewise estimate semantic entropy from multiple sampled sequences; their reported implementation samples at temperature 1 for the uncertainty estimate.[4]

This does not mean there is one universally correct temperature.

It means the sampling scheme is part of the measurement contract.

Semantic entropy

String variation can overstate uncertainty:

Paris
It is Paris.
France's capital, Paris.

Kuhn and colleagues, and then Farquhar and colleagues, address this by clustering sampled responses into semantic equivalence classes — using bidirectional entailment to decide equivalence — and estimating entropy over those meaning clusters rather than over exact strings.[4][11]

At a conceptual level:

The result is a better proxy for semantic uncertainty.

The boundary is explicit in the original work: semantic entropy targets confabulations, not every systematically wrong answer.[4]

So:

high semantic entropy
    → unstable semantic answer / confabulation risk

low semantic entropy
    → stable semantic answer
    ↛ truth

A stable misconception can remain low entropy.

13. Confidence requires three different questions

The word confidence hides several distinct measurements.

Token or sequence likelihood

The language model directly provides something like:

token likelihood
≠
correctness probability
≠
knowledge-boundary estimate
≠
verbal confidence

judge(claim)

judge(claim, evidence)

judge(claim, tools)

shared misconceptions

prompt sensitivity

domain mismatch

position bias

verbosity / surface-quality bias

authority or agreement bias

failure to inspect every claim

incorrect source interpretation

fabricated tool use if execution is not enforced

reference visible
prompt versioned
tool calls recorded
claim unit explicit
judge version recorded
blind spots evaluated

Input A:
startup has six months of runway
market is collapsing

Input B:
company has five years of runway
market is expanding rapidly

Evidence A:
Drug A increased mortality.

Evidence B:
Drug A decreased mortality.

changed output ≠ correct output

large response change can be overreaction to an irrelevant perturbation

poorly designed perturbations measure the wrong dependency

internal regime
      ↓
increased propensity for a failure behavior

model access

model-specific calibration

detector training

representation drift across versions

domain-transfer risk

pairwise closeness

evidence-set containment

truth

attribution

provenance

relational correctness

policy acceptance

Evidence:
Alice manages Bob.

Claim:
Bob manages Alice.

Evidence:
The study did not find a statistically significant reduction in mortality.

Candidate:
The study found a statistically significant reduction in mortality.

measurement_contract = {
    "sensor_id": "nli_support_v3",
    "sensor_version": "3.1.0",
    "target_property": "claim_evidence_relation",
    "measurement_unit": "claim_passage_pair",
    "granularity": "sentence_pair",
    "reference_type": "retrieved_passage",
    "output_space": {
        "type": "categorical_probabilities",
        "classes": ["entailment", "neutral", "contradiction"],
    },
    "score_direction": "class_dependent",
    "dependencies": ["claim_resolution", "evidence_retrieval"],
    "assumptions": [
        "relevant evidence has been retrieved",
        "coreferences in the claim are resolved",
    ],
    "known_blind_spots": [
        "multi-hop inference",
        "numeric range reasoning",
        "temporal ordering across passages",
    ],
    "failure_if_reference_missing": "INSUFFICIENT_REFERENCE",
    "calibration_context": {
        "dataset": None,
        "domain": None,
        "threshold": None,
        "observed_ece": None,
        "observed_brier": None,
    },
    "evaluation_hygiene": {
        "known_training_overlap": None,
        "benchmark_version": None,
    },
    "operational_metrics": {
        "latency_p95_ms": None,
        "cost_per_1000_claims_usd": None,
        "requires_gpu": None,
    },
}

entailment_probability = 0.91
runtime_state           = PASS
evidence_state          = CONFLICTING
semantic_entropy        = 1.72
source_status           = STALE
citation_support        = PARTIAL
containment_energy      = 0.08

measurements = {
    "support": {
        "sensor": "nli_support_v3",
        "value": {"entailment": 0.91, "neutral": 0.06, "contradiction": 0.03},
    },
    "runtime_state": {
        "sensor": "trace_assertion_v2",
        "value": "PASS",
    },
    "evidence_state": {
        "sensor": "source_conflict_v1",
        "value": "CONFLICTING",
    },
    "semantic_entropy": {
        "sensor": "semantic_entropy_v1",
        "value": 1.72,
    },
}

Which CVO field does this measurement improve?

What latent target does it approximate?

Which existing blind spot does it close?

What new failure modes does the measurement pipeline introduce?

What does it cost?

What authoritative check could replace it?

How will we evaluate whether it adds independent signal?

    graph LR
    WS[WORLD / SYSTEM] --> RO[references / observations]
    RO --> RC[resolved candidate]
    RC --> SE[SENSORS]
    SE --> TMR[typed measurement record]
    TMR --> CAL[calibration]
    CAL --> POL[policy + intended action]
    POL --> DEC[accept / verify / retrieve / review / abstain / reject]
  

latent property
      ≠
observable proxy
      ≠
calibrated estimate
      ≠
policy decision

What target property?
What proxy?
What reference?
What granularity?
What assumptions?
What cost?
What blind spot?

claim embedding
      ↓
project onto evidence subspace
      ↓
measure residual
      ↓
Hallucination Energy