# From Measurements to Policy
# Book: Hallucination From First Principles
# Chapter URL: https://programmer.ie/books/hallucination-from-first-principles/12-chapter/
# Prose fingerprint (sha256, normalized): 5a8ea75415958a29a4412edf1fba099fe380091cea8756307bde1834c0e93f33
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Chapter 11 ended with a typed reliability record.
It might say:
containment = PASS relation_fidelity = PASS sensitivity = PASS epistemic_adequacy = ANSWERABLE provenance = UNVERIFIED That record describes the candidate.
It still does not authorize the candidate.
To make the distinction concrete, we built a deliberately flawed reference policy that checked containment, structural fidelity, and answerability but forgot to require verified provenance for a high-risk action.
Running the same immutable record through two policy versions produced:
policy-v1 → commitment=PERMIT next_action=NONE policy-v2 → commitment=HOLD next_action=VERIFY Nothing about the candidate changed.
Nothing about the evidence changed.
Nothing about the measurements changed.
The policy changed.
That is the subject of this chapter.
Measurements describe the state of the candidate. Policy decides what that state authorizes.
Environment The small reference policy engine in this chapter was executed with:
Python 3.13.5 standard library only no external services The examples are deterministic policy tests over synthetic reliability records. They are not model-quality experiments.
Where we are The book has gradually separated concepts that are often collapsed together:
truth ≠ evidence ≠ support ≠ provenance ≠ containment ≠ structural fidelity ≠ consistency ≠ sensitivity ≠ epistemic adequacy ≠ authorization The first eleven chapters mostly asked:
What happened? What did we observe? How trustworthy is that observation? What information is missing? Chapter 12 asks a different question:
Given those observations, what is the system allowed to do?
This is where reliability becomes control flow.
1. Measurement is not diagnosis, authorization, routing, or execution Chapter 4 established that the sensor is not the verdict.
We can now make the full chain explicit:
graph TD RO[RAW OBSERVATION: energy = 0.184] --> DI[DIAGNOSTIC INTERPRETATION: containment_state = WITHIN_CALIBRATED_REGION, calibration = he-cal-v7] DI --> RR[RELIABILITY RECORD: provenance = UNVERIFIED, epistemic_adequacy = ANSWERABLE] RR --> PD[POLICY DECISION: commitment = HOLD, next_action = VERIFY] PD --> EN[ENFORCEMENT: source verification runs] EN --> ER[EXECUTION RECORD: what actually happened] These are different objects.
A diagnostic PASS means:
the sensor satisfied its own declared diagnostic contract.
It does not mean:
the application authorizes the candidate.
Likewise, low Hallucination Energy does not mean:
publish the claim send the payment commit the memory approve the deployment Strong entailment does not establish source authority.
Verified provenance does not establish freshness.
Even a completely clean reliability record may encounter a policy that prohibits the requested action.
Never let a sensor silently become a permission system.
The canonical architecture is therefore:
measurement → diagnostic interpretation → reliability record → policy → enforcement 2. There are several policy scopes The same architectural principle appears at different commitment boundaries.
Candidate policy May this claim be asserted? May this paragraph be shown? May this memory item be committed? Execution policy May this tool call execute? May this payment be sent? May this database mutation occur? Release policy May this model version reach production? May this workflow be promoted? These scopes should not automatically share schemas, thresholds, or action vocabularies.
But they share the same pattern:
evidence → structured record → policy → trusted enforcement This also clarifies how external governance work relates to this chapter. Audit-as-Code, for example, applies versioned machine-checkable gates primarily at assurance and release scope.[3] The architecture is analogous to runtime candidate policy, but the control boundary is different.
3. Use one canonical policy interface We do not need four overlapping mathematical definitions of policy.
Let:
$$ \mathcal R $$be the typed reliability record.
Let:
$$ c $$be application context:
domain risk tier action type jurisdiction latency budget human-review capacity required measurement portfolio And let:
$$ h $$be execution history:
step index retrieval attempts verification attempts clarification attempts previous routes Use two policy functions.
Commitment policy $$ \pi_{commit}(\mathcal R,c,h) \to \{\text{PERMIT},\text{HOLD},\text{DENY}\}. $$It asks:
May the candidate cross the current commitment boundary?
Commitment is the term for the assertion-scoped decision — may this content be stated. Authorization (Chapter 15) is the same kind of decision at the action boundary — may this side effect execute. They are not synonyms; they gate different boundaries, and a candidate can be committed as an assertion while the action it proposes is not authorized.
Routing policy $$ \pi_{route}(\mathcal R,c,h) \to A_{next}. $$It asks:
What should happen next?
This immediately resolves an ambiguity from simpler designs.
commitment = HOLD next_action = VERIFY means:
the candidate is not authorized yet but source verification is authorized. The old representation:
{ "authorized": False, "action": "VERIFY", } was ambiguous because VERIFY itself is an authorized action.
4. A policy decision is not one flat Action enum A single enum such as:
ACCEPT RETRIEVE VERIFY REVIEW ABSTAIN REJECT mixes different kinds of state.
ACCEPT describes commitment.
RETRIEVE describes recovery.
REVIEW describes escalation.
ABSTAIN describes response disposition.
REFUSE_REQUEST describes request-level prohibition.
These should be represented separately.
A useful policy decision has dimensions such as:
COMMITMENT PERMIT | HOLD | DENY NEXT ACTION NONE | REFINE | RETRIEVE | VERIFY | ASK | REVIEW RESPONSE MODE NORMAL | QUALIFIED | PARTIAL | ABSTENTION | REFUSAL ESCALATION AUTO | REVIEW_REQUIRED | HUMAN_APPROVAL_REQUIRED OBLIGATIONS verify_provenance state_uncertainty omit_unverified_claim ... For example:
policy_decision = { "commitment": "HOLD", "next_action": "VERIFY", "escalation": "AUTO", "response_mode": "NONE", "obligations": [ "verify_provenance", ], "reason_codes": [ "PROVENANCE_UNRESOLVED", ], } This also separates three superficially similar terminal states.
State Meaning REJECT_CANDIDATE this generated candidate is unsuitable ABSTAIN do not make the requested commitment now REFUSE_REQUEST do not perform the requested task because another policy prohibits it A hallucinated candidate can be rejected while the user request remains valid and may be regenerated.
An answer can be abstained from because evidence is inadequate.
A request can be refused even when the system knows the answer perfectly well.
Those are different failures and different control paths.
5. Hard constraints define the feasible action set Suppose a system computes:
containment = 0.95 consistency = 0.93 sensitivity = 0.90 provenance = 0.00 A weighted average could produce:
reliability = 0.695 That number can hide the only fact that matters.
If verified provenance is mandatory for this action, the candidate cannot be committed.
So policy should first derive a feasible action set.
Let:
$$ \mathcal A_{feasible}(\mathcal R,c,h) \subseteq \mathcal A $$contain only actions that satisfy all non-negotiable constraints.
Then, and only then, softer costs can choose among the remaining actions:
$$ \boxed{ a^* \arg\min_{a\in\mathcal A_{feasible}} E[L(a,Y)\mid \mathcal R,c,h] } $$
This produces the right ordering of reasoning:
HARD CONSTRAINTS What is impermissible? ↓ FEASIBLE ACTIONS What remains possible? ↓ SOFT COSTS / UTILITY Which permissible action is best? Never allow an average to cancel a non-negotiable failure.
This is the same broad pattern used in recent Audit-as-Code work: aggregate readiness can be useful while critical blockers independently force a blocking result.[3]
The policy objective is therefore not maximal restriction.
It is:
Choose the least restrictive action that satisfies the required constraints.
That preserves justified utility without permitting unjustified commitment.
6. Policy budgets should be explicit constraints Policy consumes resources:
retrieval → latency + tokens verification → compute review → human attention clarification → user effort abstention → lost coverage false acceptance → risk / harm A useful optimization can therefore be written as:
$$ \min_{\pi} E[C(\pi)] $$subject to application constraints such as:
$$ P(\text{false acceptance}\mid\pi)\le\alpha, $$$$ P(\text{mandatory under-escalation}\mid\pi)=0, $$and, where feasible,
$$ E[\mathbb I(\pi=\text{REVIEW})] \le B_H. $$Here:
$$ B_H $$is the available human-review budget.
If mandatory review demand exceeds available review capacity, the system should reduce coverage, delay, or abstain rather than silently violate the mandatory floor.
The key ordering is:
hard risk constraints first resource optimization second Chapter 6 placed cost trade-offs around detector operating points.
Chapter 12 places them where they ultimately belong: in policy.
7. Unknown states need their own type system A production record may contain:
PASS FAIL UNCERTAIN NOT_MEASURED NOT_RUN UNAVAILABLE NOT_APPLICABLE These states mean different things.
State Meaning Typical critical-field behavior PASS diagnostic contract satisfied continue FAIL diagnostic contract violated block / verify / route UNCERTAIN measurement exists but confidence is inadequate verify / review NOT_MEASURED expected measurement is absent measure / review NOT_RUN intentionally skipped because prior policy already determined route preserve as skipped UNAVAILABLE sensor could not execute abstain / escalate if critical NOT_APPLICABLE sensor has no semantic relevance legitimately ignore The classic bug is:
if state != "FAIL": allow() because then:
NOT_MEASURED becomes an accidental synonym for:
PASS. Absence of evidence from a sensor is not positive evidence of safety.
NOT_RUN is useful in progressive systems.
If a request hits a hard prohibition before an expensive sensitivity test is executed, sensitivity was not forgotten and did not fail.
It was intentionally skipped because the route was already terminal.
That distinction matters in audits.
8. Risk tiers should preserve semantics but may require more measurements Suppose:
provenance = UNVERIFIED That state should mean the same thing in every application.
But different applications may require different measurement portfolios and policies.
Exploratory drafting provenance measurement optional unverified claim may be shown with qualification Customer-facing factual answer provenance required UNVERIFIED → VERIFY Autonomous external action provenance required UNVERIFIED → HOLD or DENY So the correct rule is not:
risk tiers never change measurements.
It is:
Risk tiers should not silently change measurement semantics. They may change which measurements are required and how their states map to policy.
This preserves comparability while allowing high-risk applications to demand stronger evidence.
9. Floors apply only to ordered dimensions Some policy outputs have a natural order.
For escalation:
AUTO < REVIEW_REQUIRED < HUMAN_APPROVAL_REQUIRED Suppose policy defines a floor:
$$ r_{floor}(x). $$Then enforcement can require:
$$ r_{final}(x) \succeq r_{floor}(x). $$This gives a local architectural invariant:
the executed escalation cannot be less conservative than the deterministic minimum.
Pacella, Papadia and Giliberti demonstrate this pattern in a 2026 agentic-process experiment. Among 166 onboarding profiles, 43 required mandatory human approval; one unguarded Qwen2.5 configuration failed to escalate any of those 43 cases, while the deterministic policy floor restored mandatory-approval recall to 1.0 in the evaluated dataset and configuration.[2]
But the guarantee is local.
It assumes validated inputs, correct policy, correct extraction, and trusted enforcement.
And it applies only to ordered dimensions.
ASK RETRIEVE VERIFY are not naturally ordered.
They are different recovery branches.
Likewise:
floor compliance ≠ policy correctness. A perfectly enforced bad floor is still a bad policy.
10. Precedence chooses control; obligations preserve diagnosis Several rules may fire together.
Suppose:
policy restriction = TRUE sensitivity = FAIL The request-level prohibition should determine the terminal control action.
But the trace can still record that the sensitivity rule also fired.
A useful combination strategy is:
hard DENY overrides commitment + matched lower-priority diagnoses remain in trace + only executable obligations compatible with the terminal route survive For example:
{ "commitment": "DENY", "next_action": "REFUSE_REQUEST", "matched_rule": "POLICY_PROHIBITED", "reason_codes": [ "POLICY_PROHIBITED", "SENSITIVITY_FAIL" ], "suppressed_rules": [ "SENSITIVITY_REVIEW" ], "obligations": [] } The policy therefore does not destroy diagnostic information simply because one rule wins control flow.
This is safer than a long if/elif chain whose first branch erases every other fact about the candidate.
11. A small runnable policy engine The following reference implementation deliberately keeps the policy simple enough to inspect. It is experiments/hallucination/policy_engine.py; the routes below are its actual output.
It uses:
commitment next action escalation response mode reason codes obligations execution history rather than one flat action.
from dataclasses import dataclass from enum import Enum class Commitment(str, Enum): PERMIT = "PERMIT" HOLD = "HOLD" DENY = "DENY" class NextAction(str, Enum): NONE = "NONE" REFINE = "REFINE" # revise the candidate; executed as repair in Chapter 13 RETRIEVE = "RETRIEVE" VERIFY = "VERIFY" ASK = "ASK" REVIEW = "REVIEW" ABSTAIN = "ABSTAIN" REFUSE_REQUEST = "REFUSE_REQUEST" @dataclass(frozen=True) class History: step_index: int = 0 retrieve_attempts: int = 0 verify_attempts: int = 0 ask_attempts: int = 0 @dataclass(frozen=True) class Context: risk_tier: str = "high" max_steps: int = 4 max_retrievals: int = 1 max_verifications: int = 1 max_asks: int = 2 human_review_available: bool = True def terminal_route(ctx): if ctx.human_review_available: return "REVIEW" return "ABSTAIN" def evaluate_policy(record, ctx, history): hits = [] def hit(name, priority, commitment, action, reasons=(), obligations=()): hits.append({ "name": name, "priority": priority, "commitment": commitment, "action": action, "reasons": reasons, "obligations": obligations, }) if record.get("policy_restricted") is True: hit("POLICY_PROHIBITED", 100, "DENY", "REFUSE_REQUEST", reasons=("POLICY_PROHIBITED",)) if history.step_index >= ctx.max_steps: hit("MAX_STEPS_EXCEEDED", 95, "HOLD", terminal_route(ctx), reasons=("MAX_STEPS_EXCEEDED",)) adequacy = record.get("epistemic_adequacy") if adequacy == "USER_GAP": if history.ask_attempts >= ctx.max_asks: hit("ASK_BUDGET_EXHAUSTED", 91, "HOLD", terminal_route(ctx), reasons=("USER_INPUT_MISSING", "ASK_BUDGET_EXHAUSTED")) else: hit("USER_INPUT_REQUIRED", 90, "HOLD", "ASK", reasons=("USER_INPUT_MISSING",), obligations=("request_missing_user_field",)) if adequacy == "RECOVERABLE_EVIDENCE_GAP": if history.retrieve_attempts >= ctx.max_retrievals: hit("RETRIEVAL_BUDGET_EXHAUSTED", 91, "HOLD", terminal_route(ctx), reasons=("EVIDENCE_GAP", "RETRIEVAL_BUDGET_EXHAUSTED")) else: hit("RECOVERABLE_EVIDENCE_GAP", 90, "HOLD", "RETRIEVE", reasons=("EVIDENCE_GAP",), obligations=("retrieve_missing_evidence",)) if ctx.risk_tier == "high" and record.get("provenance") in { "UNVERIFIED", "NOT_MEASURED", "UNAVAILABLE" }: if history.verify_attempts >= ctx.max_verifications: hit("VERIFICATION_BUDGET_EXHAUSTED", 86, "HOLD", terminal_route(ctx), reasons=("PROVENANCE_UNRESOLVED", "VERIFICATION_BUDGET_EXHAUSTED")) else: hit("VERIFIED_PROVENANCE_REQUIRED", 85, "HOLD", "VERIFY", reasons=("PROVENANCE_UNRESOLVED",), obligations=("verify_provenance",)) if record.get("sensitivity") == "FAIL": hit("SENSITIVITY_REVIEW", 70, "HOLD", "REVIEW", reasons=("SENSITIVITY_FAIL",), obligations=("review_context_dependence",)) if not hits: required = { "containment": "PASS", "relation_fidelity": "PASS", "epistemic_adequacy": "ANSWERABLE", "provenance": "VERIFIED", } if all(record.get(k) == v for k, v in required.items()): hit("CLEAN_ACCEPT", 0, "PERMIT", "NONE", reasons=("ALL_REQUIRED_CONDITIONS_SATISFIED",)) else: hit("UNHANDLED_OR_UNKNOWN_STATE", 60, "HOLD", terminal_route(ctx), reasons=("REQUIRED_STATE_NOT_SATISFIED",), obligations=("inspect_unhandled_state",)) hits.sort(key=lambda x: x["priority"], reverse=True) winner = hits[0] reasons = list(dict.fromkeys( r for h in hits for r in h["reasons"] )) if winner["commitment"] == "DENY": obligations = list(winner["obligations"]) else: obligations = list(dict.fromkeys( o for h in hits for o in h["obligations"] )) return { "commitment": winner["commitment"], "next_action": winner["action"], "matched_rule": winner["name"], "suppressed_rules": [h["name"] for h in hits[1:]], "reason_codes": reasons, "obligations": obligations, "step_index": history.step_index, "policy_version": "policy-v2", } The implementation is intentionally small.
A production engine would add schema validation, policy profiles, explicit response modes, durable logs, and stronger rule composition.
But even this small engine demonstrates the chapter’s main claims.
For the tested examples it returned:
clean → PERMIT / NONE provenance = UNVERIFIED → HOLD / VERIFY policy_restricted = TRUE + sensitivity = FAIL → DENY / REFUSE_REQUEST matched = POLICY_PROHIBITED suppressed = SENSITIVITY_REVIEW recoverable evidence gap, first attempt → HOLD / RETRIEVE recoverable evidence gap, retrieval budget exhausted → HOLD / REVIEW user information gap, clarification budget exhausted → HOLD / REVIEW The important property is not that these routes are universal.
They are one explicit, testable profile.
12. Policy requires execution memory The reliability loop is stateful.
Suppose policy returns:
RETRIEVE The retriever finds nothing useful.
The system re-measures and sees the same evidence gap.
A stateless policy would return:
RETRIEVE again.
And again.
And again.
The same problem exists for:
VERIFY ASK REFINE So policy must see execution history:
$$ \pi(\mathcal R,c,h). $$Useful history includes:
step_index retrieve_attempts verify_attempts ask_attempts last_route whether state improved Then the policy can impose bounded recovery:
retrieve once ↓ no improvement ↓ REVIEW / ABSTAIN or:
ask user twice ↓ required field still absent ↓ stop asking ↓ REVIEW / ABSTAIN This second case is also a UX problem.
ASK is not a free backend action. It interrupts the user.
Repeated clarification can become clarification fatigue.
A mature router therefore has an ask budget just as it has retrieval and verification budgets.
In the reference run:
USER_GAP + ask_attempts = 0 → ASK USER_GAP + ask_attempts = 2 → REVIEW The general rule is:
A recovery route must have a stopping condition.
Without execution memory, a reliability loop can become an infinite retry loop.
13. Do not make another LLM the policy authority A common architecture is:
generator LLM ↓ reliability record ↓ small router LLM ↓ action The router may be useful as a proposal generator.
It should not be the deterministic policy floor.
If the same stochastic class of system is responsible for interpreting the rules that are supposed to constrain stochastic behavior, the control boundary becomes probabilistic again.
So distinguish:
LLM proposes route / drafts policy / explains diagnosis from:
deterministic policy engine authorizes route Learned models can absolutely contribute signals.
For example, an LLM may classify whether a source is likely authoritative.
But that output should enter the reliability record as a measurement with its own uncertainty and contract.
It should not silently become policy.
Open Policy Agent is a mature non-LLM example of the separation: structured application data is evaluated by a declarative policy decision point, while enforcement remains in the application.[4]
The model may propose. The policy engine decides. The enforcement point acts.
14. Policy as code means versioning, lineage, and replay A policy decision should have durable decision provenance.
For example:
policy_trace = { "request_id": "req_8f29", "candidate_id": "cand_3", "generator": "generator-v17", "generator_output_hash": "sha256:...", "measurement_bundle": "measurements-v8", "measurement_schema_version": "measurement-schema-v4", "policy_input_schema_version": "policy-input-v3", "evidence_snapshot_uri": "immutable://evidence/7ca...", "evidence_snapshot_hash": "sha256:...", "policy_version": "policy-v2", "step_index": 1, "matched_rule": "VERIFIED_PROVENANCE_REQUIRED", "suppressed_rules": [], "commitment": "HOLD", "next_action": "VERIFY", } A hash proves identity.
A hash alone is not a replayable evidence snapshot.
The system also needs durable immutable content, or a content-addressed reference from which the exact evidence can be recovered.
This enables two different kinds of replay.
Policy-only replay Freeze:
candidate measurement record evidence snapshot and change only:
policy v1 → policy v2 This isolates the effect of policy.
Full decision replay Freeze the original candidate and evidence, then rerun:
measurement stack v8 → v9 policy v2 → v3 This measures total evolution of the reliability stack.
Do not mix these experiments.
If a newer policy requires a field that an old measurement schema never produced, replay should not invent it.
Return something explicit such as:
POLICY_REPLAY_INDETERMINATE missing_required_field = source_freshness or re-measure from the preserved evidence snapshot.
15. Policy replay should produce a diff The deliberately flawed opening policy provides a minimal replay example.
We evaluated five fixed records under:
policy-v1 which omitted provenance entirely — both the VERIFIED_PROVENANCE_REQUIRED rule in Section 11 and the provenance == VERIFIED line in its clean-accept check — and:
policy-v2 which added both back for the high-risk profile.
The deterministic replay produced (reproduced by experiments/hallucination/policy_engine.py):
Case policy-v1 policy-v2 Changed? clean PERMIT / NONE PERMIT / NONE no provenance unverified PERMIT / NONE HOLD / VERIFY yes recoverable evidence gap HOLD / RETRIEVE HOLD / RETRIEVE no sensitivity failure HOLD / REVIEW HOLD / REVIEW no prohibited request DENY / REFUSE_REQUEST DENY / REFUSE_REQUEST no That table is more useful than saying:
v2 is safer. It tells us exactly which historical behavior changes.
A mature rollout sequence is:
graph LR HPR[historical policy-only replay] --> LSP[live shadow policy] LSP --> CCD[compare counterfactual decisions] CCD --> CE[canary enforcement] CE --> SR[staged rollout] A shadow policy reads live records and logs what it would have done without controlling execution.
Only after its disagreement set is understood should it receive enforcement authority.
16. Policy needs a set-valued oracle A policy benchmark does not always have one uniquely correct route.
Take the provenance-gap record from Section 11. On the first pass the engine routes it to VERIFY. Replay the same record with verify_attempts already at the budget and it routes to REVIEW instead. Both are correct: both hold the commitment, both carry the resolve_provenance_before_commit obligation, and which one is right depends on execution history, not on the record alone.
A test whose expected value is the single string VERIFY now fails a correct system half the time — once the budget is spent. A test whose expected value is REVIEW fails it the other half. The bug is in the oracle, not the engine.
So policy labels should define constraints instead of one exact string.
For example:
oracle = { "must_not_commit": True, "required_obligations": [ "resolve_provenance_before_commit", ], "acceptable_routes": [ "VERIFY", "REVIEW", ], } This changes what we measure.
Useful policy metrics include:
Metric Question policy violation rate did execution choose a prohibited action? mandatory escalation recall were required escalations performed? under-escalation rate was the action less conservative than required? over-escalation rate was unnecessary escalation consumed? recovery success did the route resolve the blocking condition? commitment coverage how much traffic eventually became safely commit-able? resource cost what did recovery consume? Overall route accuracy can remain a convenience metric.
It should not be the primary safety metric.
Under-escalation and over-escalation have asymmetric costs.
And:
floor compliance ≠ policy adequacy. A policy can perfectly obey an incorrect floor.
17. Progressive evaluation is an execution optimization, not an action ordering A policy engine may run cheap checks first:
graph TD HP[HARD PROHIBITIONS] --> CR[CHEAP REQUIRED FIELDS] CR --> PF[PROVENANCE / FRESHNESS] PF --> SC[STRUCTURAL / CONTAINMENT] SC --> CS[CONTEXT / SENSITIVITY] This is an evaluation order.
It does not imply:
ASK < RETRIEVE < VERIFY < REVIEW Those routes remain semantically different branches.
Progressive evaluation is useful for latency as well as cost.
Suppose a product has:
P99 latency budget = 200 ms A clean candidate may pass cheap deterministic gates in 40 ms.
A high-risk candidate may require context perturbations that exceed the synchronous budget.
The correct response is not:
latency budget exceeded → silently ACCEPT It is an explicit policy fallback such as:
HOLD ASYNC_VERIFY ABSTAIN or REVIEW according to the product contract.
A reliability policy therefore owns not only epistemic risk but operational budgets.
18. Enforcement is a separate trust boundary A prompt such as:
Only answer if every critical field is verified. is guidance.
It is not enforcement.
The stronger architecture is:
GENERATOR ↓ MEASUREMENTS ↓ RELIABILITY RECORD ↓ POLICY ENGINE ↓ POLICY DECISION ↓ TRUSTED ENFORCEMENT POINT ↓ EXECUTION RECORD Policy evaluation without trusted enforcement is advice.
A correct policy can still fail if the application ignores its output.
This gives four separate failure layers:
GENERATION FAILURE MEASUREMENT FAILURE POLICY FAILURE ENFORCEMENT FAILURE For example:
policy = DENY execution = tool_call_succeeded is not a hallucination failure.
It is an enforcement failure.
And:
Deterministic policy can be deterministically wrong.
That is why policy needs its own tests, oracles, replay, and review.
19. A minimum policy test suite A serious policy suite should contain at least:
Clean commit cases all required diagnostics satisfy contract no prohibition Expected:
PERMIT Hard prohibition cases Expected:
DENY + REFUSE_REQUEST Recoverable evidence gaps Expected constraint:
must_not_commit acceptable route:
RETRIEVE Missing user-owned variables Expected:
HOLD + ASK until the ask budget is exhausted.
Unknown critical measurements Expected:
never implicit PERMIT Conflicting rules Example:
policy_restricted = TRUE sensitivity = FAIL Expected:
DENY + REFUSE_REQUEST matched_rule = POLICY_PROHIBITED suppressed_rules includes SENSITIVITY_REVIEW Recovery-loop tests RETRIEVE → no improvement → retrieve budget exhausted → terminal REVIEW / ABSTAIN Floor property tests For ordered escalation coordinates:
$$ r_{final}\succeq r_{floor} $$for every generated test case.
Replay schema tests Historical record missing a newly required field:
→ POLICY_REPLAY_INDETERMINATE not implicit pass.
20. Worked question: why are NOT_MEASURED, NOT_RUN, and NOT_APPLICABLE different? Suppose a regulated reporting policy requires verified provenance.
Case A — NOT_MEASURED The provenance sensor was expected to run but no result exists.
meaning: required evidence about provenance is missing route: VERIFY / MEASURE / REVIEW Case B — NOT_RUN A hard legal prohibition already produced a terminal refusal before provenance measurement was needed.
meaning: measurement intentionally skipped after route was determined route: keep terminal refusal Case C — NOT_APPLICABLE The candidate is a purely creative fictional passage for which provenance is outside the task contract.
meaning: provenance has no semantic relevance route: ignore this field Treating all three as None destroys precisely the information policy needs.
Typed missingness is part of the control architecture.
21. Using AI to help write policy Section 13 said not to make an LLM the policy authority. This section is about letting an LLM help with policy authorship. Those are different: authorship produces candidate rules that are then translated to explicit semantics and tested; authority is the run-time decision, which stays deterministic.
An LLM can be useful during policy development.
Ask it to:
propose missing edge cases generate synthetic policy records suggest reason codes translate prose requirements into candidate rules review a replay diff But do not give the generated policy enforcement authority merely because the prose looks plausible.
A useful adversarial exercise is:
Draft a reliability policy for a high-risk factual system. Then test it on: provenance = NOT_MEASURED containment = PASS epistemic_adequacy = ANSWERABLE If the generated router effectively implements:
if state != "FAIL": allow() it has failed the chapter’s most basic invariant.
The AI can help write the test.
The deterministic test decides whether the policy passes.
22. What you should now be able to answer After this chapter, you should be able to explain:
Why raw measurement, diagnostic state, authorization, routing, and execution must remain separate. Why a policy decision needs commitment, recovery, escalation, obligations, and reason codes rather than one flat action enum. Why hard constraints should define the feasible action set before cost optimization occurs. Why NOT_MEASURED, NOT_RUN, UNAVAILABLE, and NOT_APPLICABLE have different policy meanings. Why policy floors apply only to genuinely ordered dimensions. Why recovery routes require execution history and bounded retry budgets. The difference between policy-only replay and full decision replay. Why a correct policy without trusted enforcement is not a control. 23. Exercises Exercise 1 — Separate policy dimensions Rewrite this flat result:
action = VERIFY as a typed decision containing:
commitment next_action escalation response_mode obligations reason_codes Explain what each dimension contributes.
Exercise 2 — Test unknown-state behavior Create one critical field in each state:
PASS FAIL UNCERTAIN NOT_MEASURED NOT_RUN UNAVAILABLE NOT_APPLICABLE Specify the allowed policy behavior for each.
Verify that only NOT_APPLICABLE can be ignored semantically and that NOT_RUN is justified by an earlier terminal route.
Exercise 3 — Test a policy floor Define:
AUTO REVIEW_REQUIRED HUMAN_APPROVAL_REQUIRED Generate random proposed and minimum escalations.
Assert:
$$ r_{final}\succeq r_{floor}. $$Then explain why the same property cannot be applied to ASK versus RETRIEVE.
Exercise 4 — Bound a recovery loop Create a record with:
epistemic_adequacy = RECOVERABLE_EVIDENCE_GAP Run policy before and after the retrieval budget is exhausted.
Expected:
first → RETRIEVE after budget → REVIEW / ABSTAIN Exercise 5 — Replay a policy change Create five fixed measurement records.
Change one rule:
high-risk provenance UNVERIFIED v1 → PERMIT v2 → VERIFY Produce a diff table and explain which behavior changed without regenerating any candidate.
Exercise 6 — Separate policy and enforcement failure Construct:
policy commitment = DENY execution status = COMPLETED Identify the failed layer and the telemetry needed to prove it.
24. The deeper lesson The first half of this book was largely about observation.
We learned:
fluency is not truth retrieval is not support support is not provenance containment is not entailment consistency is not correctness sensitivity is not adequacy confidence is not answerability Now we can add:
Evidence is not authorization.
And we can make the statement stronger:
$$ \boxed{ \text{measurement} \neq \text{diagnosis} \neq \text{authorization} \neq \text{routing} \neq \text{execution} } $$A reliable system preserves those boundaries in its types, not merely in its documentation.
The model may propose.
The measurements diagnose.
The policy constrains.
The enforcement layer acts.
graph TD GEN[GENERATOR] --> MEAS[MEASUREMENTS] MEAS --> REC[RELIABILITY RECORD] REC --> PD[POLICY DECISION] PD --> ENF[ENFORCEMENT] ENF --> ER[EXECUTION RECORD] The policy decision becomes operational only when enforcement records what action was actually authorized and executed.
And because recovery is stateful, the loop can continue:
graph LR M1[measurement] --> P1[policy] --> RET[RETRIEVE] RET --> NE[new evidence] --> RM[re-measure] RM --> P2[policy] --> VER[VERIFY] VER --> NE2[new evidence] --> RM2[re-measure] RM2 --> P3[policy] --> PER[PERMIT] or terminate safely when recovery stops making progress.
This is the point where reliability stops being commentary on model behavior and becomes part of the software architecture.
Generation may be stochastic. Acceptance does not have to be.
Research roots Chloe Autio, Reva Schwartz, Jesse Dunietz, Shomik Jain, Martin Stanley, Elham Tabassi, Patrick Hall and Kamie Roberts, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, 2024; NIST publication page updated in 2026. Provides voluntary lifecycle-oriented guidance for managing generative-AI risk in context rather than prescribing one universal threshold. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
Massimo Pacella, Gabriele Papadia and Vincenzo Giliberti, “Governed Agentic Process Automation: A Floor-Safety Guarantee for Compliance-Critical LLM Routing,” Algorithms 19(8), 627, 2026. Formalizes a closed routing space, policy-derived minimum escalation, deterministic fallback, and local floor-safety guarantee; evaluates the architecture on 166 onboarding profiles. https://www.mdpi.com/1999-4893/19/8/627
Aoun E. Muhammad, Kin-Choong Yow and Shrooq Alsenan, “Audit-as-code: a policy-as-code framework for continuous AI assurance,” Frontiers in Artificial Intelligence, 2026. Applies versioned machine-checkable governance gates, evidence bundles, risk-tier thresholds, and non-overridable blockers primarily at assurance / release scope. https://doi.org/10.3389/frai.2026.1759211
Open Policy Agent Project, “Open Policy Agent Documentation,” current documentation. Describes a general-purpose declarative policy engine that evaluates structured inputs separately from the application’s enforcement point. https://www.openpolicyagent.org/docs
Next: Verification, Repair, and Rejection Chapter 12 can now express routes such as:
HOLD + RETRIEVE HOLD + VERIFY HOLD + REFINE DENY + REJECT_CANDIDATE But those are still instructions.
They need execution semantics.
If policy says:
VERIFY what evidence should be gathered?
If it says:
REFINE how do we repair a response without silently changing its meaning?
If a repair produces a new candidate, does the old policy decision still apply?
And when should the system stop trying and reject the candidate?
Chapter 13 turns policy routes into concrete recovery loops.
The next question is:
Once the system distrusts a candidate, how should it recover without creating a new failure?