Part 2 — Get the Model Out of the Chat Box

Three numbers that will not add

Three calls in Chapter 12 performed the same small task, through the same gateway, and all three succeeded. One route reported 38 input tokens. Another reported 279. The third reported 73.

Put those in a spreadsheet and the next move writes itself: sum them, average them, divide the invoice by them, route to whichever chamber is “cheapest per token”. Every one of those moves assumes the three numbers measure the same thing.

They do not, and the reason is not that one of the providers is wrong.

In 1999 NASA lost the Mars Climate Orbiter. Its investigation board found the root cause was the “Failure to use metric units in the coding of a ground software file” (NASA, 1999). A program called SM_FORCES produced thruster impulse data in English units. The interface documentation required metric, so the trajectory modelers built to that requirement. Nothing crashed; the numbers flowed, were accepted, were used, and over nine months put the spacecraft about 170 kilometers lower than planned.

The field existed on both sides of the interface. The unit did not travel with it.

Which usage quantities are actually equivalent enough to normalize, and what should a system do with the ones that are not?

A name is not a unit

Stevens defined measurement as the assignment of numerals to objects or events according to rules, and showed that the rule decides which operations on the numerals mean anything (Stevens, 1946). Two numbers produced under different rules cannot be added just because they are both integers, and they certainly cannot be added because they share a field name.

Token usage is a textbook case. The field names are shared across API dialects. The rules behind them are not. Here is what the two documented dialect families say, taken from their own documentation:

Sources: OpenAI’s prompt-caching and reasoning guides (OpenAI); Anthropic’s prompt-caching and extended-thinking documentation (Anthropic). Both were read on 13 September 2026, and both change.

Read the first row again. The same field name, input_tokens, has opposite relationships to cached input in the two dialects. In one, the cache is inside the number. In the other, it is beside it. Code that reads input_tokens from both and adds them is not normalizing. It is doing exactly what the orbiter’s ground software did.

The last row makes the point sharper still. Even inside one dialect, a cached token and an uncached token are priced differently, by an order of magnitude or more. A “total tokens” figure was never a cost proxy, even before you cross a dialect boundary.

Four rules for a field

So normalization is a decision, made per quantity. CodeAI’s usage interpreter applies four rules, and they are worth stating plainly because every system that consumes usage makes these choices somewhere, usually by accident:

1. Equivalent: normalize. A route’s reported prompt count and another route’s input count are the same kind of quantity (tokens the model processed as input) and can share a canonical component, provided the dialect’s rule for what is inside that number is known.
2. Related but not equivalent: preserve separately, with the relation declared. Cache reads are inside input in one dialect and added to input in another. They get their own component, tagged subset_of_input or additive_to_input. Reasoning gets its own component, tagged subset_of_output. Nothing is summed until the relation says it may be.
3. Unknown semantics: preserve raw, report UNKNOWN. A field with no rule, such as audio_tokens inside a Chat usage detail, is kept by path and not interpreted, not dropped, and never added.
4. A provider’s total is an observation, not an answer. A reported total_tokens is kept as reported and compared with the derived total. If they disagree, both survive and the disagreement is recorded as a conflict. Neither replaces the other.

And one rule that runs through all four, from Chapter 11: absence is not zero. When a component is not reported, a derived total is UNKNOWN. If some parts are known, the interpreter records a lower bound, which is honest and useful, and is not a total.

From those rules the interpreter derives three quantities, each under a named formula:

In code, the decision lives in a small table: for each dialect, where each component is found and what relation it has to its parent. Reduced from CodeAI’s usage interpreter:

_RULES = {
    "responses": {
        "input_includes_cache": True,
        "input":          (None, "input_tokens", PRIMARY),
        "cache_read":     ("input_tokens_details", "cached_tokens", SUBSET_OF_INPUT),
        "reasoning":      ("output_tokens_details", "reasoning_tokens", SUBSET_OF_OUTPUT),
        "total_reported": (None, "total_tokens", REPORTED_TOTAL),
        ...
    },
    "chat_completions": {   # rule_source: Responses semantics "applied by analogy ... (not separately verified)"
        "input_includes_cache": True,
        "input":          (None, "prompt_tokens", PRIMARY),
        "cache_read":     ("prompt_tokens_details", "cached_tokens", SUBSET_OF_INPUT),
        ...
    },
    "messages": {
        "input_includes_cache": False,
        "input":          (None, "input_tokens", PRIMARY),
        "cache_read":     (None, "cache_read_input_tokens", ADDITIVE_TO_INPUT),
        "total_reported": None,
        ...
    },
}

Every rule also records its rule_source, so an interpretation can say which documentation it relied on, and the Chat rule’s source admits in its own text that it is an analogy. A protocol with no entry gets no derived quantities at all; the interpreter reports no_semantic_rule_for_protocol rather than guessing.

The experiment: same bytes, two interpretations

This is the Stage 13 build, and it needs no new model calls. The raw material is the response bytes Chapter 12 already preserved: three live captures, one per dialect, plus the decoded payload of Chapter 11’s truncated call.

Two interpreters read those bytes. usage-semantics-v1 reproduces the historical adapter view exactly: two numbers, input and output, extracted as the adapters always extracted them. usage-semantics-v2 applies the four rules. Both are pure functions of the preserved observation. Neither writes anything, and the runtime’s recorded usage and past decisions are untouched.

Before any code ran, the expected v2 result for every case was written by hand from the dialect documentation, and that file’s hash is recorded in the report. Then the harness ran with outbound sockets refused:

Alongside those four real cases, a synthetic specification corpus of twenty cases covered edge conditions outside the four captures: absent usage, partial usage, measured zeros, a conflicting reported total, cache reads inside and beside input, and cache writes without reads. The remainder stressed reasoning inside output, an estimate, a gateway cost field, an unrecognized detail, malformed counts such as negative, string and null values, subsets larger than their parents, and a protocol with no rule at all.

All 24 cases matched the hand-written expectations. Every source byte hash was identical before and after. No network calls were made. The same projection was then run through Runtime.interpret_usage_as over copies of the actual recorded Stage 12 attempts. It read the preserved transport bodies, appended no ledger events, changed no artifacts, and left each attempt’s recorded usage in its historical two-number form.

What the numbers say once they are interpreted

Look at the fresh-input column.

Interpret the numbers within each route first. Responses reported no cache component, so its fresh-input projection remains 38. Chat reported 192 cached tokens inside its 279-token input count, so v2 derives 87 fresh-input tokens for that route. Messages defines input_tokens as excluding cache, so its reported 73 is already the fresh-input component even though total input remains UNKNOWN because the cache fields were absent from the payload.

Chapter 11’s different MiMo request also reported exactly 192 cached tokens. The repeated count suggests a stable route-level cached prefix, but the record holds only the count, not the cached bytes for comparison. More importantly, 38, 87 and 73 are now internally clearer measurements; they still are not a cross-model ranking because the routes do not establish a shared token unit.

Now look at what did not become a number. Under the Messages rule, cache reads and writes are added to input. The MiniMax route left both fields out of the payload. The interpreter keeps 73 as a floor with lower bounds of 73 and 293 for total input and processed total. v1 would have given you 73 and let you believe it was the whole input. v2 gives you 73 as a floor and says so.

That is the difference between a normalizer and a renamer. A renamer maps prompt_tokens to input_tokens and moves on. An interpreter asks what is inside the number first.

What v2 caught that v1 could not

Four things surfaced that the historical view had no way to express.

A reported zero beside visible reasoning content. Both MiMo calls reported reasoning_tokens: 0 while returning reasoning content: 780 characters in Chapter 12, 1,128 characters in Chapter 11. v2’s Chat rule treats reasoning tokens as a subset of output by analogy to the documented OpenAI usage semantics; Chapter 13 does not claim that this OpenCode route is a conforming OpenAI implementation. The interpreter therefore does not “correct” the zero, because it has no evidence for the right count. It records reasoning_tokens_zero_with_reasoning_content, preserves the reported zero, and marks route conformance unverified. The payload does not fit the assumed usage mapping cleanly on either call; that is evidence to retain, not a number to repair.

Content with no count. MiniMax returned a 1,094-character thinking block and no thinking_tokens field. v2 marks reasoning as not reported, not zero.

Silent coercion in the old path. Fed a count of -5, v1 recorded −5. Fed the string "100", v1 recorded 100. That is what the adapters’ historical extraction does: it coerces whatever it finds into an integer. v2 marks both values invalid, derives only a lower bound from what remains, and records which path was wrong. A bad count that becomes a plausible number is Chapter 12’s silent-configuration debt arriving in the measurement layer.

Unrecognized detail and gateway cost. The Chat routes reported audio_tokens in both detail objects. v2 keeps them by path and interprets nothing. All four responses carried a gateway field "cost": "0", a string. v2 preserves it with its path and marks accounting as unknown. On a subscription plan (Chapter 6), a per-call cost of zero is not a price.

Even interpreted, tokens are not a shared unit

It would be tempting to read 38, 87 and 73 as three comparable fresh-input counts. They are not, and this is the limit of normalization.

Each count came from a different route serving a different model. The preserved responses establish what each route reported; they do not establish that all three routes used the same tokenizer, the same preprocessing, or even an identical definition of the token unit beyond the dialect semantics already examined. Petrov and colleagues showed how sharply tokenized length can vary across tokenizers, including disparities of up to fifteen times across languages (Petrov et al., 2023). That result is a warning about assuming commensurability, not proof of which tokenizer OpenCode used on these three calls.

So normalization makes a route’s usage internally coherent. It can say what is inside each reported quantity, which parts are cached, which are reasoning, and which are unknown. It cannot establish that two routes’ token counts share one unit without additional evidence about the tokenization and preprocessing behind those routes. That is why Chapter 10 compared chambers on cost per passing item rather than cost per token. Here, token counts are route-local measurements.

Observation, interpretation, accounting

The chapter’s architecture is three layers, and v2 deliberately stops at the second:

observation     the preserved response bytes, content-addressed         never rewritten
     ↓
interpretation  usage-semantics-v2: components, relations, derived      versioned, replaceable
                values, conflicts, diagnostics, UNKNOWNs
     ↓
accounting      a bill, a quota draw-down, a budget charge              needs a pricing shape

Accounting would need things the interpreter does not have and should not guess: the pricing shape of the plan (Chapter 6: a subscription with usage caps is not a per-token rate), the multiplier for each component (cache reads and writes are priced differently from fresh input and from each other), and an effective date for the rates. None of that was built. cost stays UNKNOWN until a separately justified pricing interpretation exists.

The same architecture as preserved bytes plus replaceable readings:

    flowchart TD
    RAW["raw response bytes<br/><i>content-addressed, never rewritten</i>"] --> V1["interpretation v1<br/><i>kept: history decided under it</i>"]
    RAW --> V2["interpretation v2<br/><i>current: components · relations · UNKNOWNs</i>"]
    RAW -.->|"new docs, new evidence"| V3["interpretation v3<br/><i>projection over the same bytes</i>"]
    V1 --> R["canonical runtime result"]
    V2 --> R
    V3 -.-> R
    R -.->|"stops here: needs a pricing shape"| AC["accounting<br/><i>not built</i>"]
    style AC stroke-dasharray: 4 4
  

The versioning is what makes this safe. v1 is not deleted, because decisions already made were made under it. The retries and call statuses from Chapters 11 and 12 remain bound to the interpretation in force when they were decided. v2 is a projection over the same observation. When a dialect’s documentation changes, or when a route is shown not to follow its dialect’s rule, the fix is v3 over the same bytes, and history does not move.

Where it is still weak

1. The Chat rule is an analogy. OpenAI’s documentation establishes the relations for Responses. v2 applies them to Chat detail fields by analogy, and says so in its rule source.
2. Route conformance is unverified everywhere. These are OpenAI- and Anthropic-shaped dialects serving MiMo, GPT and MiniMax models through OpenCode. A dialect rule describes what an API family documents; it does not prove that a gateway route implements those semantics exactly. The two MiMo responses — visible reasoning content beside a reported reasoning count of zero — show that the route does not fit the assumed mapping cleanly. That is enough to keep conformance unverified; it is not enough to diagnose exactly where the mismatch originates.
3. The real corpus is thin. Four real responses, two from the same model. None exercised a Messages cache read, a non-zero reasoning count, a cache write, or a conflicting total. Those rules are tested only synthetically.
4. Nothing consumes v2 yet. Recorded attempts, call totals and experiment budgets still use the two-number v1 view, so a budget still counts input plus output, with cached prefixes included. The recorded path has a failure of its own that no interpretation can repair: Chapter 28 found an adapter that discarded usage the provider had reported whenever a response arrived with no text, before anything downstream could read it.
5. A missing Messages cache report blocks a total. That is correct, and it means budgets on Messages routes lose precision until either the route reports cache fields or a documented omission convention justifies treating absence as zero.
6. The consistency check is narrow. It catches a zero reasoning count only when reasoning content is visible in the response. An under-reported non-zero count, or reasoning a route does not return, passes unnoticed.
7. Unrecognized-path detection looks one level deep, and estimates are carried but never produced.
8. The preserved Stage 13 run was made from a dirty CodeAI working tree. Its manifest pins base commit cfd02e0, the modified and untracked Stage 13 paths, and the exact source hashes used by the run. The Stage 13 implementation was subsequently committed in 862d64d, but the evidence bundle itself remains a dirty-tree execution. A clean rerun from the committed implementation is preserved separately in usage-semantics/offline-862d64d/: no dirty paths, source hashes identical to the dirty-tree run, and identical results. The original bundle is kept as it was.

Do this now

Forty minutes. Find out what your token numbers contain.

1. For each dialect you consume, fill in the relation table from its documentation: is cached input inside or beside input_tokens, is reasoning inside output, is a total reported? If you cannot fill a cell, that relation is currently a guess in your code.
2. Take your last twenty usage records. Count how many carry cache, reasoning or detail fields that your code ignores or adds.
3. Search your code for the point where “not reported” becomes 0. There is almost always one.
4. Take any two routes you compare and compute fresh input for each under its own dialect’s rule. Does the ranking you were using survive?

If you are building with an assistant:

Add a versioned usage interpretation over preserved provider responses.
Do not change recorded usage, past decisions or stored bytes.
- Keep the historical two-number view as v1, reproduced exactly.
- In v2, read each component by documented path: input, output, cache read,
  cache write, reasoning, reported total. Tag each with its relation
  (subset of input, additive to input, subset of output) per dialect rule.
- Derive total input, fresh input and processed total only when every needed
  component is reported and valid; otherwise UNKNOWN with a lower bound.
- Keep a reported total and compare it with the derived one; record conflicts.
- Preserve unrecognized fields by path. Never coerce strings, negatives or
  nulls into counts.
- Write expected results by hand from the documentation BEFORE running, then
  replay real stored responses and a synthetic corpus, checking that source
  hashes are unchanged. No network calls. No billing.

Failure modes

• Normalizing by field name. input_tokens has opposite relations to cached input in two major dialects.
• Adding nested details to their parents. Cached and reasoning tokens counted twice.
• Filling “not reported” with zero. A lower bound silently becomes a total.
• Replacing a provider’s total with your derivation, or the reverse. Keep both and record the conflict.
• Coercing bad counts. -5 and "100" became plausible numbers in the historical path.
• Believing a provider’s breakdown. Zero reasoning tokens arrived twice beside visible reasoning.
• Comparing token counts across models. Different tokenizers, different units, even after normalization.
• Treating tokens as cost. Cached and uncached tokens are priced an order of magnitude apart.
• Overwriting the old interpretation. Decisions made under it lose their basis.

What this chapter established

• Three successful calls reported 38, 279 and 73 input tokens for one request. Like the Mars Climate Orbiter’s thruster data, the field traveled across the interface and the unit did not.
• Measurement is numerals assigned by a rule, and the rule decides which arithmetic means anything. OpenAI Responses counts cached input inside input_tokens; Anthropic Messages counts it beside. The same name has opposite semantics.
• Four rules per quantity: normalize the equivalent; preserve the related-but-not-equivalent with a declared relation; keep unknown semantics raw and UNKNOWN; keep a reported total as an observation and record conflicts. Absence gives a lower bound, never a total.
• Built in CodeAI as usage-semantics-v2, a pure, versioned projection over preserved bytes, with v1 reproducing the historical view. Four real responses and twenty synthetic cases matched hand-written expectations, with source hashes unchanged, no network calls, no ledger writes and recorded usage untouched.
• Interpreted route by route, the measurements become more meaningful: Responses projects 38 fresh input; Chat separates 192 cached tokens from 279 reported input to derive 87 fresh input; Messages treats its 73 reported input as fresh while leaving total input UNKNOWN because cache components were absent from the payload. These are route-local interpretations, not a cross-model ranking.
• v2 surfaced what v1 could not: reasoning counts of zero beside reasoning content on two calls, content with no count, silent coercion of invalid values, unrecognized fields, and a gateway cost string that is not a price.
• Normalization makes a route’s usage internally coherent. It does not establish that different routes or models share one token unit; that requires separate evidence about their tokenization and preprocessing semantics.
• Observation, interpretation and accounting are separate layers. No bill was built, and none can be without a pricing shape.

Next

Chapter 11’s offline trace ended on a line this part of the book has been carrying ever since: task_status: not automatically completed.

Every chapter since has made the call more truthful. It records what was intended, preserves what came back, survives a change of dialect, and now knows what its usage numbers contain. None of that says whether the work is done. A complete, correctly accounted, perfectly interpreted answer is still only a candidate. Something has to decide whether it satisfies the task, and it cannot be the model.

Continue with The Model Is Not the Process.

References

• S. S. Stevens. On the Theory of Scales of Measurement. Science, vol. 103, no. 2684 (1946), pp. 677–680. https://doi.org/10.1126/science.103.2684.677
• Mars Climate Orbiter Mishap Investigation Board. Phase I Report. NASA, 10 November 1999. https://llis.nasa.gov/llis_lib/pdf/1009464main1_0641-mr.pdf
• Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, and Adel Bibi. Language Model Tokenizers Introduce Unfairness Between Languages. Advances in Neural Information Processing Systems 36 (NeurIPS), 2023. https://proceedings.neurips.cc/paper_files/paper/2023/hash/74bb24dca8334adce292883b4b651eda-Abstract-Conference.html
• OpenAI. Prompt caching and Reasoning models guides. Accessed 13 September 2026. https://developers.openai.com/api/docs/guides/prompt-caching, https://developers.openai.com/api/docs/guides/reasoning
• Anthropic. Prompt caching and Extended thinking documentation. Accessed 13 September 2026. https://platform.claude.com/docs/en/docs/build-with-claude/prompt-caching, https://platform.claude.com/docs/en/docs/build-with-claude/extended-thinking

Implementation sources: the preserved Stage 13 evidence run was executed from CodeAI base commit cfd02e0 with the Stage 13 implementation still present as modified and untracked working-tree files; experiments/applied-ai/evidence/usage-semantics/offline/manifest.json pins those dirty paths and the exact source hashes used by the run. The Stage 13 implementation was subsequently committed in CodeAI 862d64d (chapter 14 setup), and experiments/applied-ai/evidence/usage-semantics/offline-862d64d/ reruns the harness from that clean commit with identical source hashes and results. Relevant symbols are interpret_usage, USAGE_SEMANTICS_V1, USAGE_SEMANTICS_V2, Component, Derived, UsageInterpretation, and Runtime.interpret_usage_as; tests and harness are tests/test_usage_semantics.py and experiments/usage_semantics_demo.py. Evidence: experiments/applied-ai/evidence/usage-semantics/ (expected.json authored before the run, offline/ report and table, runtime-projection.json). Source observations: experiments/applied-ai/evidence/protocol-conformance/live/ and experiments/applied-ai/evidence/ch11-live-opencode/.