The Verifier Passed. The Experiment Was Still Wrong.

What ZeroModel learned when evidence integrity, scientific validity, and visual-policy meaning came apart.
Introduction
The first ZeroModel article showed that a bounded policy could be compiled into a durable artifact and read without invoking its source model at every decision. But the runtime was given something enormously valuable for free: the exact state address.
The next experiment removed that shortcut. We gave the runtime only the rendered screen and asked the observation itself to locate the correct policy row.
The exact version worked. Then we introduced variation, ambiguity, rejection, and time.
What began as an experiment in visual policy lookup became an experiment in proving what our own experiment actually knew.
[PLACEHOLDER: Introduce the defining reversal. We built an adversarial verifier that detected every corruption we asked it to detect. External review then found that several benchmark constructions did not instantiate the phenomena their names claimed.]
[PLACEHOLDER: Once today’s repairs land, add one sentence explaining that the eventual correction did not erase this failure. It turned the failure into the principal result.]
1. The Sign Had an Address
In Signs, Not Directions, the runtime received a symbolic state such as:
tank=3|target=5|cooldown=0
That string was an exact address into the compiled policy artifact.
symbolic state
↓
policy row
↓
candidate action values
↓
selected action + trace
The policy compilation worked. The exhaustive verification worked. The runtime could trace every decision back to one exact row, one exact cell, and one exact artifact identity.
But it did not have to discover where it was.
[PLACEHOLDER: Briefly restate the first article’s validated results. Confirm the final visual-decision count against the repository before publication.]
[PLACEHOLDER: Explain the relationship to VIPER in no more than two paragraphs. VIPER extracts and verifies the policy; the new ZeroModel programme concerns the observation that addresses it.]
The unresolved question was therefore not:
Given the state, what should the system do?
It was:
What does the observation justify the system in claiming its state to be?
2. Can the Screen Become the Address?
The arcade world renders to a small grayscale frame. Its policy-relevant state is encoded visually through the tank position, target position, and cooldown indicator.
ZeroModel compiled a second artifact:
rendered frame
↓
visual feature codeword
↓
visual index artifact
↓
policy row ID
↓
compiled policy artifact
↓
action + trace
The visual index addresses the policy without becoming the policy.
That separation matters. Observation representation and decision semantics have different identities. A visual provider can be replaced without rewriting the policy. A policy can change without silently inheriting an old visual index.
[PLACEHOLDER: Add a concrete visual-index identity and its declared addresses relationship to the policy artifact.]
[FIGURE PLACEHOLDER: The two-artifact architecture—visual index on the left, compiled policy on the right.]
This was not general visual understanding. It was a bounded visual address that was permitted to refuse.
3. A Complete Canonical Victory
The canonical reader worked across the complete finite state universe.
canonical policy rows recovered: 112 / 112
canonical actions recovered: 112 / 112
four-target waves cleared: 2,401 / 2,401
visual decisions equivalent to
the symbolic policy path: [CONFIRM FINAL COUNT]
Every canonical state produced a distinct visual codeword. Collisions were rejected during compilation. Unknown and corrupted fixtures were refused instead of being forced into the nearest row. Serialization preserved artifact identity.
For a moment, the result appeared to give the second article a simple conclusion:
The screen could replace the symbolic state address.
It could—but only inside the exact canonical visual universe.
[FIGURE PLACEHOLDER: A representative sample of the 112 canonical rendered states.]
4. The Codebook Boundary
The first important correction came from measuring how far an observation could move while remaining accepted.
The reader’s acceptance radius was smaller than one integer feature step.
pixels change,
quantized feature codeword unchanged
↓
same canonical address
feature codeword changes
↓
rejection
The system tolerated changes that disappeared beneath the representation’s own quantization. It did not tolerate meaningful feature variation.
The honest result was therefore:
The canonical Visual Sign Reader implemented exact feature-codeword addressing with refusal—not tolerant recognition of visually varied observations.
That was still a real result. It was simply narrower than the phrase “visual reader” initially suggested.
[PLACEHOLDER: Insert the exact calibration fact and the test or artifact that established it.]
5. The Foundation Model Wins the Wrong Metric
The next phase tested whether more approximate representations could tolerate benign visual changes while rejecting observations that should not address the policy.
The providers included:
exact deterministic codewords
normalized pixels
DINOv2 medoid retrieval
DINOv2 all-prototype retrieval
ridge row classification
DINOv2 performed competitively on action accuracy while performing badly on exact state identity and rejection.
[RESULT PLACEHOLDER]
| Provider | Exact-row accuracy | Action accuracy | False-acceptance rate |
|---|---|---|---|
| Normalized pixels | [VALUE] | [VALUE] | [VALUE] |
| DINOv2 all prototypes | [VALUE] | [VALUE] | [VALUE] |
[PLACEHOLDER: Retain the “correct actions from the wrong policy row” result after confirming its denominator and final wording.]
The representation was often finding a state that looked semantically similar and happened to prescribe the same action. That could look successful under an action-only metric while failing the addressing contract.
The conclusion was not that foundation models are bad at vision.
It was narrower:
Semantic visual similarity and governed policy addressing are different objectives.
6. The Right Action from the Wrong Row
The apparent failure contained a more interesting signal.
A policy maps many states onto a smaller number of actions. Several visually confused rows may still agree:
row A ─┐
row B ─┼──→ RIGHT
row C ─┘
Other ambiguities are genuinely conflicting:
row A ───→ LEFT
row B ───→ RIGHT
These cases cannot be reported as the same kind of error.
Historical experiments showed a substantial gap between exact-row identification and action identification.
[RESULT PLACEHOLDER]
| Provider | Exact-row top one | Action top one | Difference |
|---|---|---|---|
| System B | [VALUE] | [VALUE] | [VALUE] |
| Registered local correlation | [VALUE] | [VALUE] | [VALUE] |
The result suggested that visual representations could lose exact state identity while preserving substantial policy information.
That did not make a wrong row correct.
It raised a new question:
Could the system preserve the complete ambiguity set and act only when every supported row agreed?
7. The Question Our Evidence Could Not Answer
We returned to the historical experiment artifacts looking for the answer.
They contained:
top-one row
top-one action
aggregate counts
They did not contain:
all row scores
complete rankings
explicit numerical ties
candidate-set identities
frame-level beliefs
temporal composition traces
The winner had been saved. The belief surface that produced it had been discarded.
As a result, the historical artifacts could not tell us whether a fixed-top-k set, a score-gap set, a conformal set, or an action-unanimous set would have been safe or useful.
[PLACEHOLDER: Add the exact retrospective audit status and the experiments it covered.]
This became one of the project’s clearest lessons:
Evidence preservation is not bookkeeping. It determines which future questions the experiment remains capable of answering.
8. Preserve the Whole Belief Surface
The next benchmark was designed prospectively. Before running it, ZeroModel specified the evidence every observation/provider pair had to retain.
all policy-row raw scores
all quantized scores
canonical row universe
complete ranking
explicit tie groups
semantic top-set status
provider identity
policy identity
observation and episode identity
transformation and seed lineage
temporal composition trace
The resulting structure, CompleteRowEvidence, preserves the complete row-score surface rather than only its winner.
Identity-bearing quantities are reconstructed from deterministic, quantized evidence. Raw floating-point scores remain available for diagnostics without defining canonical identity.
[PLACEHOLDER: Explain why this distinction matters for reproducibility and later optimized implementations.]
[FIGURE PLACEHOLDER: One observation expanding into the complete policy-row score surface, ranking, tie groups, and semantic outcome.]
The new instrument could preserve the evidence required to ask the candidate-set question.
It had not yet proved that the evidence was meaningful.
9. A Tie Is Not a Winner
A deterministic ordering is useful for writing rows to disk. It is not evidence that one tied row was observed.
The repaired semantic outcome distinguishes:
unique_row
action_unanimous_tie
conflicting_action_tie
unresolved
A unique row resolves a state.
An action-unanimous tie may resolve an action while refusing to invent a state.
A conflicting-action tie resolves neither.
[PLACEHOLDER: Add the canonical diagnostic demonstrating how lexical top-one reporting differed from semantic top-set reporting. Recheck the figures after today’s changes.]
The deeper point is simple:
Alphabetical order can serialize uncertainty. It cannot remove it.
[FIGURE PLACEHOLDER: An equal-score top group, contrasted with a falsely selected lexical winner.]
10. Can Time Remove the Ambiguity?
A single frame may support several rows. A sequence may support fewer.
The intended temporal operation is:
previous candidate rows
+
actual executed action
↓
action-conditioned reachable rows
∩
current visual candidates
↓
narrowed candidate set
The temporal layer is only allowed to remove possibilities. It must not insert a state that the current observation does not support.
The action must also be the action actually executed—not the action the candidate policy row says should have been executed.
Otherwise the system uses its own possibly mistaken belief to confirm its next belief.
[PLACEHOLDER: Once repaired, explain the new action-conditioned transition artifact and how it differs from the earlier row-union tile.]
[PLACEHOLDER: Add one real sequence showing candidate-set narrowing, or explicitly state that utility remains unevaluated if no prospective run has occurred.]
[FIGURE PLACEHOLDER: Candidate sets across two or three frames, narrowed by the executed action.]
11. The Experiment Begins Testing Itself
Before materializing the prospective benchmark, ZeroModel built an adversarial verifier.
The verifier independently reconstructs:
evidence identities
row universes
rankings
tie groups
episode plans
seed derivations
transformations
negative-family contracts
control-family contracts
reachability traces
access restrictions
It then applies deliberate mutations and checks that each corruption is rejected for the expected reason.
Some mutations also recompute the affected digest. These “digest laundering” cases ensure that the verifier is not merely trusting an artifact’s own internally consistent story.
[PLACEHOLDER: Insert the final mutation count after today’s repairs. Distinguish integrity mutations, phenomenon mutations, provider-conformance mutations, and semantic invariants.]
The verifier passed every corruption we had asked it to detect.
That was the problem.
We had not yet asked it the most important questions.
12. The Invalid Frame That Was Valid
One negative benchmark family attempted to create an impossible observation by combining visual regions from two states that prescribed conflicting actions.
The contract checked that:
the source actions conflicted
the result differed from source A
the result differed from source B
both sources contributed pixels
the mask and identities reconstructed
But the arcade universe contains every valid combination of tank position, target position, and cooldown.
Replacing the complete target region from one valid state with the target region from another did not necessarily create an invalid observation.
It could create a third valid state.
State A
tank 2, target 1, cooldown 0
State B
tank 5, target 6, cooldown 1
splice:
tank and cooldown from A
target from B
result:
tank 2, target 6, cooldown 0
↓
valid State C
The verifier proved that the result differed from its parents.
It did not prove that the result lay outside the universe of valid observations.
[PLACEHOLDER: After today’s repair, insert the committed exhaustive collision test demonstrating that the old construction fails and the new invalid family is globally outside the canonical universe.]
[FIGURE PLACEHOLDER: State A + State B → State C. This should be the article’s principal visual.]
The error was a mismatch between a local construction property and a global scientific property:
Showing that an output differs from its inputs does not establish that it differs from everything legitimate.
13. The Same Error in Several Forms
The splice was the clearest example, but the same boundary appeared elsewhere.
Action-labelled reachability
The earlier reachability tile placed action labels on edges whose destination sets did not actually depend on the action.
[PLACEHOLDER: Explain the repair and the exhaustive test over all row/action pairs.]
Prescribed versus executed action
Temporal composition consulted what the policy prescribed rather than what the environment actually executed.
[PLACEHOLDER: Explain how executed-action identity is now carried through the temporal state.]
Information-theoretic controls
A valid ambiguity control must contain distinct hidden histories that produce the same provider-visible observation.
[PLACEHOLDER: Explain the corrected control construction and its hidden-history cardinality requirement.]
Provider development evidence
A provider cannot claim stability under variation when its development pairs compare each prototype only with itself.
[PLACEHOLDER: Explain the repaired P3 development fixture or the decision to keep P3 unavailable.]
Reference versus optimized implementations
An optimized path cannot establish equivalence by delegating directly to the reference path.
[PLACEHOLDER: State whether an independent optimized path now exists or remains explicitly unavailable.]
These were not five unrelated bugs.
They were five examples of the same mistake:
A structure can resemble the required experiment without causally implementing it.
14. Repairing the Instrument
[THIS SECTION ASSUMES TODAY’S REPAIRS LAND.]
The repair did not discard the evidence instrument. Most of the instrument was doing exactly what it was designed to do.
The project retained:
complete score evidence
quantized identity
explicit tie semantics
causal seed lineage
sealed episode plans
bounded caches
independent reconstruction
mutation-based verification
final-split access restrictions
The repair strengthened the layer those mechanisms had not yet covered.
Global phenomenon checks
[PLACEHOLDER: Describe exhaustive valid-universe collision testing for negative observations.]
Genuine action-conditioned transitions
[PLACEHOLDER: Describe the authoritative transition source and the full row/action audit.]
Actual action lineage
[PLACEHOLDER: Describe how the executed action is recorded, carried, and verified.]
Distinct hidden histories
[PLACEHOLDER: Describe the corrected information-theoretic control.]
Real provider conformance
[PLACEHOLDER: Describe the independent recomputation of provider outputs from the provider formula.]
Lowest-level access enforcement
[PLACEHOLDER: Describe the final-materialization guard at the lowest materialization boundary and any denied-access ledger.]
The important result was not merely that the code was fixed.
It was that the verifier’s definition of correctness had expanded.
15. Three Proofs for Visual Policy
The programme arrived at three independent proof obligations.
1. Artifact integrity
Are the identities, score vectors, rankings, tie groups, transformations, and traces internally consistent and independently reconstructible?
2. Phenomenon validity
Does the benchmark example genuinely instantiate the phenomenon it claims to measure?
Is the invalid observation actually invalid? Does the action-conditioned edge actually depend on the action? Does the ambiguity control contain multiple hidden histories?
3. Provider conformance
Did the declared provider formula actually produce the recorded evidence?
Was the real scorer executed? Did an independent implementation reproduce the result? Was a synthetic fixture substituted for the provider under test?
artifact integrity
+
phenomenon validity
+
provider conformance
↓
defensible scientific claim
All three are necessary. None substitutes for another.
[FIGURE PLACEHOLDER: The three-proof model as either three sequential gates or a triangle.]
These categories do not exhaust scientific validity.
They are the three that ZeroModel’s own failures forced it to name.
16. What ZeroModel Can Say Now
[PLACEHOLDER: Replace this table after the corrected instrument lands and its final status is known.]
| Claim | Publication status |
|---|---|
| Bounded symbolic policy compilation | Validated |
| Exact canonical visual addressing | Validated within canonical scope |
| Historical action-level signal | Measured and bounded |
| Complete evidence preservation | Implemented and verified |
| Semantic candidate-set mechanics | Implemented |
| Negative-family phenomenon validity | [POST-REPAIR STATUS] |
| Information-control validity | [POST-REPAIR STATUS] |
| Action-conditioned temporal reachability | [POST-REPAIR STATUS] |
| Provider conformance | [POST-REPAIR STATUS] |
| Prospective candidate-set utility | [NOT RUN / RESULT] |
| Independent optimized path | [UNAVAILABLE / VERIFIED] |
| Final evaluation | [NOT RUN / RESULT] |
| Production readiness | Unsupported unless independently established |
The exact canonical reader remains real.
The historical action-level signal remains interesting.
The prospective instrument preserves evidence that the historical experiments lost.
[PLACEHOLDER: State the strongest final claim supported by the repaired repository—no stronger.]
17. What Happens Next
Once the instrument satisfies all three proof obligations, ZeroModel can finally run the experiment it originally wanted to run:
Can complete visual candidate sets preserve useful policy information under ambiguity, and can temporal evidence safely narrow those sets without fabricating certainty?
The relevant measurements will not be headline action accuracy alone.
They will include:
ground-truth row coverage
unique-row resolution
action-unanimous resolution
conflicting-action rejection
candidate-set size
temporal narrowing
unsafe acceptance
invalid-family rejection
control-family non-identifiability
provider-conformance failures
[PLACEHOLDER: Insert the actual prospective experiment status at publication time.]
The candidate-set hypothesis may succeed.
It may fail.
The important change is that the next result will mean what its label says.
Coda
The first ZeroModel article argued that a mature system should recognize when a route has become stable enough to place a sign.
This experiment added the harder half.
A system must establish which sign the observation justifies it in reading. An experiment must establish that its evidence is intact, that its examples genuinely represent the declared phenomena, and that its real providers generated the evidence being judged.
The verifier passed because it answered every question we had taught it to ask.
The experiment was still wrong because the most important questions had not yet been encoded.
[PLACEHOLDER: Once the repair is complete, add one final sentence acknowledging that the purpose of the verifier was not defeated by this discovery; it was completed by it.]
A mature system should not merely place signs. It should be able to prove that it is standing at the junction the sign was built for.