Part 2 — Get the Model Out of the Chat Box

The smallest working call

CodeAI’s OpenCode cognition adapter, invoked directly without the recorded-call runtime, does the minimum well. One CallSpec becomes one user message, the message goes out as an ordinary HTTP request, and a CallResult comes back:

adapter = OpenCodeCognitionAdapter(model="mimo-v2.5", protocol="chat_completions")
result = adapter.invoke(spec)  # one CallSpec in, one CallResult back
if result.status != "succeeded":
    raise RuntimeError(result.error)
review = result.raw_output

It keeps failure in its own channel instead of passing a diagnostic off as a review. It returns the usage with its source label alongside the convenient string. Those two habits already put it ahead of a great deal of production code.

Now ask it the questions Chapter 10 said every call must answer. Which gateway served this? Which model, at which revision? What did it cost, under which price table? How many attempts did it take? What exactly came back before anything parsed it — and where is that stored now that the process has exited?

The string answers none of them. The dictionary answers some, until the process ends.

What is the smallest model call that is still useful for everything Part 1 asked of it?

The answer is larger than a function that returns text, and the size of that gap is the subject of this chapter. A string is the smallest working call. The smallest useful call is a recorded process event.

Why “useful” costs more than “working”

Every requirement below was argued earlier and is now due.

None of that is intelligence. All of it is bookkeeping around one stochastic step, and all of it is deterministic: Chapter 3’s rule, applied to the call itself.

Three identities that must never collapse

task_id  ≠  call_id  ≠  attempt_id

Each answers a different question.

• Task — what are we trying to get done? Review paragraph P for unsupported claims.
• Call — what did we decide to ask, of which chamber? Ask deep-review to review P.
• Attempt — what happened on the wire, this time? Sent at 07:49:33; answered 21.8 seconds later.

Collapse them and specific things become impossible to say. Here is the defect that shows why, in a single-result client that is otherwise good. When an explicit output limit is set and the first response comes back empty, the client retries once, at twice that limit. The caller invokes invoke() once and receives one result. Two requests reached the provider; one usage figure came back. A cost computed from that result understates the work, and nothing in the returned value says so.

A retry is not a new task and not a new call. A retry is another attempt beneath the same call. Once attempts exist as records, “how many times did we actually ask?” has an answer that does not depend on reading adapter source.

The three identities as a recorded flow, in write order:

    flowchart TD
    T["task<br/><i>what are we trying to get done?</i>"] --> C["call<br/><i>what did we decide to ask, of which chamber?</i>"]
    C --> M["manifest<br/><i>intent, frozen before any attempt</i>"]
    M --> A1["attempt 1<br/><i>what happened on the wire, this time</i>"]
    A1 --> O1["raw observation 1<br/><i>preserved before parsing</i>"]
    A1 -.->|"retry: another attempt<br/>beneath the same call"| A2["attempt 2"]
    A2 --> O2["raw observation 2"]
    O1 --> CC["call.completed"]
    O2 --> CC
  

Intent before effect

The central move in CodeAI’s recorded call is to write down what it intends before anything happens, and to record what it observed separately.

Immediately before a request leaves, CodeAI freezes a manifest: the chamber, the logical model requested, the occupant it resolved to, the pricing version in force, the prompt and context identity, and both the parameters the caller asked for and the parameters that will actually be sent. Each attempt then records reality: when it started and finished, the provider’s request id, what status came back, what usage was reported, and a pointer to the preserved raw observation.

A simplified extraction of the two records:

@dataclass(frozen=True, slots=True)
class CallManifest:            # intent: written before any attempt
    call_id: str
    task_id: str
    chamber: str | None
    requested_model: str | None
    provider: str | None             # the gateway, e.g. "opencode"
    resolved_model_id: str | None
    provider_revision: str | None    # None means unknown, never a timestamp
    pricing_version: str | None
    prompt_hash: str | None
    requested_parameters: Mapping[str, Any]
    effective_parameters: Mapping[str, Any]

@dataclass(frozen=True, slots=True)
class AttemptRecord:           # observation: one per try
    attempt_id: str
    call_id: str
    task_id: str
    attempt_index: int
    protocol: str | None             # wire dialect, kept apart from the gateway
    provider_request_id: str | None
    status: str
    error_kind: str | None
    usage: Usage                     # counts may be None, with a source label
    cost_usd: float | None
    raw_artifact: ArtifactRef | None # content-addressed, stored before parsing
    normalizer_version: str | None

This is not a new idea dressed up for AI. The W3C provenance model separates the entities that exist, the activities that generate and use them, and the agents responsible, and relates them with terms such as used and wasGeneratedBy (Moreau & Missier, 2013). Read the recorded call in those terms. An attempt is an activity; it used the manifest; the raw observation is an entity that wasGeneratedBy that attempt; the interpreted result is a further entity derived from the observation. Keeping those links is what lets a later process ask which observation produced which conclusion.

The write order is the design

CodeAI appends events in a fixed order: call.requested, call.manifest, then for each attempt attempt.started, the provider request, the raw artifact, attempt.completed, and finally call.completed.

The order matters because of what happens when a process dies in the middle. Birrell and Nelson, describing one of the first practical remote procedure call systems, stated the guarantee plainly. If a remote call returns, the procedure ran precisely once. If an exception is reported instead, it ran “either once or not at all—the user is not told which” (Birrell & Nelson, 1984). A model call over HTTP inherits exactly that uncertainty. The gateway may have served the request and counted it; your process may never know.

CodeAI cannot remove that ambiguity, and the runtime’s own contract says so: it promises clean-restart durability, not exactly-once execution. What writing intent first buys is narrower and valuable. If the process dies after attempt.started, the ledger holds an attempt with no completion — a visible ambiguity instead of a silent loss. An interrupted call is always inspectable as unresolved, because its manifest was written before anything was attempted.

A constructed crash window makes the states concrete — fake adapter, no network, same identities as the trace above:

crash after call.manifest, before attempt.started
  ledger: call.requested, call.manifest — no attempt
  safe reading: no provider effect was possible; starting is safe

crash after attempt.started, before any completion
  ledger: call.requested, call.manifest, attempt.started — no attempt.completed
  safe reading: the provider may or may not have served it; reconcile from
  the gateway (or a later reading of its books) before deciding, never repeat blindly

The second row is the one to internalize. It is not a failure with a retry button; it is a question the ledger leaves open on purpose, with the intent still readable beside it. A later part of the book gives that question a name and a procedure; until then, “unknown outcome, do not repeat automatically” is the complete discipline.

Four states that are not the same

The rule that runs through the whole implementation:

unknown  ≠  absent  ≠  zero  ≠  inferred

Concretely:

Zero is a measurement. Treating “nothing was reported” as zero is not a harmless default, and statistics has understood why for fifty years. Rubin showed that ignoring the process that causes data to be missing is safe only under specific conditions — broadly, that whether a value is missing does not depend on what the missing value would have been (Rubin, 1976). Token usage fails that condition. It goes missing most often on failed attempts: timeouts, rejected requests, dropped connections. Fill those gaps with zero and every cost-per-call figure is biased downward — worst exactly where failures are most frequent, which is exactly where Chapter 6’s invoice most needs to be right. That application to token accounting is this book’s argument, not Rubin’s result; the principle it rests on is his.

Two consequences in the implementation are worth noticing.

Call totals go unknown if any attempt is unknown. If attempt 1 reported no usage and attempt 2 reported 264 input tokens, the call’s total input is recorded as unknown rather than 264. That is honest, and also lossy: “at least 264, completeness unknown” would preserve more without lying. The current choice is the conservative one, and Chapter 13 should revisit it when it designs usage interpretation.

A number the gateway states is not automatically a price. The live OpenCode response below includes a field "cost": "0". CodeAI does not import it. OpenCode Go is sold as a subscription — $10 a month after a $5 first month — with usage caps of $12 per five hours, $30 per week and $60 per month (OpenCode). Under that model a per-call cost of zero plausibly means “no marginal charge”, not “consumed nothing”. Until its meaning is established, the honest record is unknown. The larger lesson for Chapter 6 is that pricing has a shape, not just a rate. Under a quota, the scarce resource is headroom against the cap, and a per-token price table cannot represent it.

An offline trace, run end to end

What follows was executed for this chapter against CodeAI at commit 2327838, with the fake adapter: no network, no credentials. The fixture’s “review” is a canned string. It is evidence that the runtime records execution, and no evidence of model ability whatsoever.

Three separate processes: create a task, record one call through the deep-review chamber, then reopen the ledger and inspect the call.

$ codeai run create "Review paragraph P for unsupported factual claims" \
    --success "every flagged claim names a checkable source"
run_id: b464dd30-ccee-4d67-855b-b030ea41537a
task_id: 64883b78-fc50-4840-aa87-06b86b1d72f3

$ codeai call --task 64883b78-… --chamber deep-review --logical-model review \
    --prompt "Review this paragraph …" \
    --response "The one-second latency claim requires measurement."
call_id: b2a5c12e-120d-479e-a51d-91e1b07174e3
chamber: deep-review
requested_model: review
provider_revision: UNKNOWN
pricing_version: 2026-09-01
attempt 1: id=7ad5f5f2-bc6b-4497-a001-7d00663e5287 status=succeeded
  raw_artifact: e75d23fa88618ac369653efc87343fe4c39bdf5acec7c43d775971f7e2ece453
  usage_source: measured cost=None
call_status: succeeded
task_status: not automatically completed

$ codeai calls show b2a5c12e-120d-479e-a51d-91e1b07174e3      # a new process
call_id: b2a5c12e-120d-479e-a51d-91e1b07174e3
attempt 1: id=7ad5f5f2-bc6b-4497-a001-7d00663e5287 status=succeeded
  raw_artifact: e75d23fa88618ac369653efc87343fe4c39bdf5acec7c43d775971f7e2ece453
call_status: succeeded

The identities survive a process boundary. The revision is unknown and says so. The cost is None because the fake model has no price, not because it was free. And the last line of the second command is the one Chapter 14 is built on: the cognition step succeeded, and the task is not complete.

A second constructed run exercises two attempts under one call. Attempt 1 returns an empty body and reports no usage; attempt 2 returns text with 264 input and 41 output tokens. The caller also asks for a reasoning_effort setting the fake provider does not support.

call_id: call-001      task_id: task-001      chamber: deep-review
requested_parameters:  {temperature: 0.2, reasoning_effort: low, max_tokens: 256}
effective_parameters:  {temperature: 0.2, max_tokens: 256, model: mimo-v2.5}

attempt 1  transient_failure  error_kind=empty_output  usage=unavailable    raw=e4a89f4a…
attempt 2  succeeded                                   usage=264/41 measured raw=52bdc0c1…

call_status: succeeded      total_input_tokens: unknown      total_cost: unknown

Same task, same call, two attempt identities, two preserved observations. The unsupported setting is visibly absent from what was sent, which is Chapter 12’s whole subject arriving early. One detail deserves a flag. The fake adapter reported attempt 1 as succeeded with no text; the runtime reclassified it as a transient failure and retried. That reclassification is a policy decision, and we will return to it.

Four live calls, and what the record caught

Then the same path ran against the real gateway: OpenCode, model mimo-v2.5, through CodeAI’s recorded runtime rather than an ad-hoc request. The ledgers from those runs were written to temporary directories. They have since been exported, scanned for credentials, and preserved with this book under experiments/applied-ai/evidence/ch11-live-opencode/.

It is tempting to tell this as “four attempts: three failures, then success.” That would break the definition this chapter just gave. These were four separate recorded calls in four separate workspaces, each with exactly one attempt, with CodeAI’s code changed between them. They are experiments, not retries.

Run 4 recorded everything this chapter set out to capture: gateway opencode, protocol chat_completions, model mimo-v2.5, revision unknown, provider request id gen-1789285773-…, 264 input and 256 output tokens measured, latency 21.8 seconds, cost unknown, and the raw observation stored under sha256:1a459bea….

The flattering reading — that once execution is recorded, failures stop being mysteries and become evidence — holds, because each failed run pointed directly at the next fix.

The more useful reading is less comfortable. The record’s observations were accurate. Several of its interpretations were not.

• Run 1 was labeled an authentication failure. The response body said it was an edge rejection; the credential was never checked. The classifier maps 401 and 403 to “authentication” from the status code alone.
• Run 3 was labeled a provider error. A 400 says the request was at fault — ours.
• Run 4 was labeled a success. Open its preserved observation and it reports "finish_reason": "length". The model used all 256 output tokens it was allowed, and the review it returned ends mid-sentence:

This statement contains an **unsupported factual claim** because it makes an
absolute guarantee ("every request within one second") without providing
evidence, context, or qualifications.

**Why it's unsupported:**
- **Absolute language** ("every

At that commit, CodeAI read no finish or stop reason anywhere, so a truncated answer and a complete one were recorded identically.

Every one of those errors was found in the same place: the preserved raw observation, read after the fact. Had CodeAI stored only its interpretation — authentication_error, provider_error, succeeded — the misreadings would be permanent and invisible. Because observation was kept apart from interpretation, the interpretation can be corrected without rewriting history. That is precisely why each attempt carries a normalizer_version, and it is the problem Chapter 13 exists to solve.

The same observation holds one more fact the current interpretation ignores: 192 of the 264 input tokens were served from cache. Whether that changes the cost depends on the gateway’s semantics — again, Chapter 13.

What this chapter deliberately leaves out

The recorded call is the substrate. The chapters that follow each take one part of it seriously:

• Chapter 12 — why requested and effective parameters differ, and how one operation survives several wire dialects without leaking them into the runtime. The live runs already showed that the gateway and model can be right while the protocol is wrong.
• Chapter 13 — who decides what usage, cost and status mean, and how an interpretation is versioned and replaced without touching the observation.
• Chapter 14 — who owns the work once a call succeeds, since success here means only that a cognition step produced output.
• Chapter 22 — retries as policy: when repeating a request is safe, and what happens to side effects.

Where it is still weak

The implementation earns the claims made above and no more. These are the gaps an honest reader of the code finds, in rough order of consequence:

1. Completion is not recorded. No finish or stop reason is read, so truncated output is indistinguishable from complete output.
2. Classification is unversioned interpretation. Status codes and message fragments become error kinds with no version attached, and two of the four live runs were misclassified. Treating an empty “success” as a retryable failure is also a policy, and some chambers legitimately return nothing.
3. Not every call path is recorded. The sealed-fanout path, which carries experiment and arm identities, still uses the older call method and produces no attempt records — so Chapter 10’s per-chamber release protocol cannot yet run on it.
4. Effective-parameter sanitizing is a denylist. It drops any key with a credential-like word as one of its parts. Tested directly, it removes max_token and token_limit, and it would remove a real control such as prompt_cache_key — so the record of what was sent can omit something that was sent. An allowlist of known controls, plus a recorded list of what was redacted, would fail safe in both directions.
5. Redaction happens before preservation. “Raw” means sanitized decoded JSON, not bytes and headers. A sanitizer bug destroys evidence permanently, and header-only fields are never kept.
6. Identity is only unique within a ledger. All four live runs used the hand-assigned call id call-live-1. Merge their ledgers and the identities collide.
7. Ambiguity is recorded but not reported. An orphaned attempt.started is inspectable, yet nothing lists orphans for a person to resolve.
8. Cost cannot represent a subscription. The price table models per-token rates; the gateway the book uses is priced by quota, so every real call so far records cost as unknown.

These gaps are stated as they stood at 2327838, and several close later in the book, each with its own evidence. Chapter 12 reads finish and stop reasons and preserves the exact response bytes of gateway calls rather than sanitized JSON. Its versioned interpretation, extended in Chapter 17, can recognize the Cloudflare block that Run 1 misread. Chapter 16 projects an orphaned attempt.started as an effect of unknown outcome instead of leaving it for someone to stumble on. By Chapter 23 the fan-out path runs through the recorded call. The credential denylist and subscription-shaped cost remain as described.

None of these is a reason to distrust what the record says it observed. Each is a place where what it concludes should be held more loosely than it currently is.

Do this now

Forty-five minutes, with a coding assistant. Make one call a recorded event.

Give your assistant this prompt. The full version, with acceptance evidence, is prompt 2 in docs/applied-ai/reader-prompts.md. Two lines ask for things CodeAI itself does not yet do — recording truncation and an allowlist — because you should build the corrected version.

In this repository, make one model call a recorded process event.
Inspect first: trace the current call path and report what is recorded today.
Then implement, reusing existing types where possible:
- separate task_id, call_id and attempt_id; a retry is a new attempt, never a new task;
- write a manifest (chamber, requested and resolved model, gateway, protocol,
  pricing version, prompt hash, requested vs effective parameters) BEFORE the request;
- record each attempt (timing, provider request id, status, usage with a source
  label, raw observation stored content-addressed BEFORE parsing);
- keep unknown as unknown: no zero-filled usage, no invented revisions, no zero cost;
- read the provider's finish or stop reason and record truncation;
- never persist credentials; capture effective parameters with an allowlist.
Prove it offline with a fake adapter: one call with two attempts, reopened in a
second process. Do not add routing, evaluation or retry policy.

Then answer these from the running system, not from the assistant’s report:

1. Kill the process between the request and its completion. Can you find the orphaned attempt?
2. Pick one recorded status. Can you open the raw observation behind it and show whether the status is right?
3. Is there any path in your code that still calls a model without producing an attempt record?

Failure modes

• Returning a string and calling it a call. The smallest working call answers none of the questions measurement needs.
• Collapsing task, call and attempt. Retries vanish into single results and costs are understated silently.
• Recording only after success. A crash mid-call leaves no trace instead of a visible ambiguity.
• Zero-filling unknowns. Missing usage clusters on failures, so zero biases cost downward exactly where it matters.
• Inventing a revision. A timestamp is not a model version.
• Importing a gateway’s number without its meaning. "cost": "0" under a subscription is not a price.
• Storing only the interpretation. Misclassifications and truncations become permanent and undiscoverable.
• Trusting the classifier. “succeeded” meant “produced text”, including text cut off mid-sentence.
• Calling separate experiments attempts. It flatters the story and breaks the definition the accounting depends on.
• Leaving one call path unrecorded. The unrecorded path is the one your experiments will quietly use.

What this chapter established

• A string is the smallest working model call; the smallest useful one is a recorded process event, because Part 1’s demands — reproducibility, a bounded blast radius, an honest invoice, paired measurement, comparable occupants — are all bookkeeping around the call.
• task ≠ call ≠ attempt. A retry is another attempt beneath the same call. A single-result client shows the defect this prevents: two provider requests, one result, one usage figure.
• Intent before effect. A manifest records what was resolved and intended before the request; attempts record what was observed. In provenance terms the observation wasGeneratedBy the attempt, which used the manifest.
• A remote call that fails may have run once or not at all. Writing the manifest and attempt.started first turns that into a visible ambiguity rather than a silent loss — clean-restart durability, not exactly-once execution.
• unknown ≠ absent ≠ zero ≠ inferred. Missing usage clusters on failed attempts, so zero-filling biases cost where accuracy matters most. A gateway’s "cost": "0" under a subscription is not a price, and quota-shaped pricing cannot be represented by a per-token table.
• Executed offline: one call recorded and reopened across three processes with identities intact and the task still incomplete, and a two-attempt call in which unknown usage propagates to an unknown total and an unsupported setting is absent from the effective request.
• Recorded live, and preserved: four separate OpenCode calls, three failures then one success. The observations were accurate while the interpretations were not — a Cloudflare block labeled authentication, a client-side 400 blamed on the provider, and a truncated answer labeled a success. All three were caught in the preserved raw observations.
• That substrate is real and still weak in named places: completion state, versioned classification, the unrecorded fanout path, a denylist sanitizer, redaction before preservation, ledger-local identity, unreported orphans, and cost for subscription gateways.

Next

Look again at the second and third live runs. The gateway, the model, and the credential were all right. What was wrong was the dialect — the shape of the request and response for that particular model — and then a header that dialect required.

The recorded call made that visible. It did not make it go away. The same logical operation, “ask deep-review to review paragraph P”, has to be expressed differently depending on which route the chamber’s occupant is served on, and the next occupant may be served on another. The runtime should not have to know.

Continue with One Operation, Several Model APIs.

References

• Andrew D. Birrell and Bruce Jay Nelson. Implementing Remote Procedure Calls. ACM Transactions on Computer Systems, vol. 2, no. 1 (1984), pp. 39–59. https://doi.org/10.1145/2080.357392
• Donald B. Rubin. Inference and Missing Data. Biometrika, vol. 63, no. 3 (1976), pp. 581–592. https://doi.org/10.1093/biomet/63.3.581
• Luc Moreau and Paolo Missier (eds.). PROV-DM: The PROV Data Model. W3C Recommendation, 30 April 2013. https://www.w3.org/TR/2013/REC-prov-dm-20130430/
• OpenCode. OpenCode Go — model endpoints and pricing. Accessed 2026-09-13. https://opencode.ai/v2/docs/console/go

Implementation sources: CodeAI at commit 2327838 — src/codeai/domain.py (CallManifest, AttemptRecord, RecordedCall, Usage, UsageSource, CostSource), src/codeai/runtime.py (invoke_recorded_call, _classify_attempt, _aggregate_totals), src/codeai/adapters.py (sanitize_effective_params, NORMALIZER_VERSION, CognitionAdapter, CallResult), src/codeai/providers.py (OpenCodeCognitionAdapter, OPENCODE_ENDPOINTS, _classify_opencode_error, _chat_text). Live evidence — experiments/applied-ai/evidence/ch11-live-opencode/. Review of the implementation and its direction — docs/applied-ai/reviews/ch11-technical-direction-review.md.