CodeAI

The problem

Calling a model is easy.

The hard part begins immediately afterwards.

A real application has to know:

• what the model was asked to do;
• which context it actually received;
• which call and attempt produced an output;
• whether the transport succeeded and whether generation completed;
• which claims came from that output;
• what evidence supports those claims;
• whether the system is allowed to change anything;
• whether a requested effect actually happened;
• whether the resulting state passed an adequate check;
• whether the task may be accepted as complete;
• whether a retry is safe;
• and what operation should happen next.

A chat window answers many of those questions implicitly because a person is holding the process together.

That stops working when the model becomes one component inside software.

The central problem of Applied AI is therefore not:

how do I call a model?

It is:

how do I put intelligence inside a process
without making the process itself stochastic?

Why the obvious approach fails

The obvious architecture is an agent loop:

prompt
  -> model
  -> tool
  -> model
  -> tool
  -> done

That can be useful, but the diagram hides the most important questions.

Who decided that the work was done? Was a tool call merely requested, or was its effect observed? Did a verifier inspect the exact state produced by the action? Is the verifier independent of the model that proposed the answer? Did a retry repeat an external effect? Was a later call allowed to see a sibling model’s output? Did the system choose another model because the task required more cognition, or simply because the first call failed?

If those distinctions live only in the prompt or transcript, they are conventions rather than runtime properties.

CodeAI makes them explicit.

The solution

CodeAI is the working runtime behind Applied AI.

Its core rule is:

the model supplies cognition
the runtime owns durable state

claims are not evidence
capability is not authority
requested effects are not observed effects
verification is not acceptance
a successful call is not completed work

choose the next operation before choosing the model

The authoritative record is an append-only event ledger. Model outputs, response bodies and other evidence are preserved as artifacts. Higher-level views — claims, process state, cost views, decisions and task completion — are projections over that durable record.

The runtime separates five jobs that agent loops often blur together:

1. Cognition — a model proposes or interprets.
2. State — the ledger records what the process can actually establish.
3. Authority — a recorded grant determines which effects are permitted.
4. Verification — checks gather evidence about an artifact or observed state.
5. Policy — deterministic code decides which epistemic operation comes next.

The model participates in the process. It does not own the process.

How it works

The architecture is built around explicit boundaries:

human intent
    -> directive + bounded grant
    -> task + success criteria
    -> deterministic context compilation
    -> recorded model call
    -> preserved transport observation
    -> versioned interpretation
    -> claims + evidence
    -> deterministic process-state projection
    -> next-operation decision
         CALL | CHECK | ASK_HUMAN | STOP

separate effect path:
    recorded authority
    -> requested action
    -> effect attempt
    -> runtime observation
    -> bound verification
    -> acceptance
    -> completion

everything durable
    -> append-only ledger + content-addressed artifacts
    -> replay and reprojection

A few boundaries are especially important.

Context is selected, not remembered

A model receives a compiled context package rather than whatever happens to be in a conversation window. The package can be hashed, budgeted, inspected and replayed. Isolation seals can keep sibling outputs out of independent calls.

Observations come before interpretations

Provider bytes, transport outcomes and artifacts are preserved before the runtime decides what they mean. Interpretation is versioned, so a later rule can reinterpret the same evidence without rewriting history.

Effects have their own boundary

A model may be capable of proposing a change without being authorized to cause it. Actions are checked against recorded authority, idempotency and preconditions. The runtime distinguishes the worker’s report from its own observation of resulting state.

Verification binds to what was actually checked

A PASS is useful only when the system can say what artifact or state the verifier examined. Verification can therefore support acceptance without becoming a generic statement that the whole task is correct.

Completion is derived, not declared

A task is not complete because a call returned successfully or an agent wrote “done”. Completion projects from a validated acceptance record and its causal completion event.

The scheduler chooses an operation, not an answer

Current CodeAI projects a ProcessState from durable events, then passes those facts to a pure deterministic scheduler.

The policy can return:

CALL
CHECK
ASK_HUMAN
STOP

Process completion and process-budget exhaustion stop the process. An unresolved external effect escalates to a person. Owed verification runs before more cognition. Model budget can block another CALL without blocking a CHECK.

The scheduler does not authorize effects. Authority remains a separate path.

Evidence and validation

CodeAI is deliberately an evidence-bearing project rather than a clean architecture diagram.

The book’s capstone first asked whether one small task could move through the whole process and leave enough durable evidence for another reader to reconstruct what happened.

That path exercised:

directive
-> task
-> compiled context
-> model call
-> claim
-> verification
-> authorized effect
-> runtime observation
-> verification of the resulting state
-> acceptance
-> completion
-> replay without a second effect

The capstone also exercised stop branches for denied authority, stale state, conflicting idempotency, verifier error, uncertain effects and failed verification.

Then the composition audit asked a harder question:

Is each arrow merely visible in the record, or does the runtime actually enforce it?

The frozen Wave 1 audit classified thirteen joints. It found that reconstruction held while several important relationships were still only recorded or conventional. Among the gaps were acceptance authority, decision-to-execution binding, effect observation, artifact binding and claim-side verification trace.

Those failures were not removed from the evidence.

They became the next engineering work.

Follow-on experiments and current source strengthen specific seams, including:

• acceptance authority resolved from the task’s recorded directive rather than a caller’s claimed grant;
• process state projected from ledger events before scheduling;
• scheduler decisions recorded with their state snapshot and basis events;
• decision-to-operation binding available to governance paths;
• stronger effect-observation semantics;
• artifact/check binding;
• explicit acceptance basis when an accepted result is tied to an action;
• durable trace of verification attempts against claims.

That progression is part of the proof. The system did not begin correct because the architecture sounded sensible. Its joints were measured, failures were preserved, and repairs earned stronger claims one seam at a time.

Capstone and implementation

In Applied AI, CodeAI is the cumulative implementation of the book’s central argument:

A model gives you intelligence. Applied AI is the engineering required to make that intelligence participate reliably in a process.

The main implementation surfaces are:

src/codeai/runtime.py          runtime orchestration and durable operations
src/codeai/context.py          deterministic context compilation
src/codeai/process_state.py    task state projected from the ledger
src/codeai/scheduler.py        deterministic next-operation policy
src/codeai/authority.py        recorded grants and effective authority
src/codeai/actions.py          effects, idempotency and recovery state
src/codeai/verification.py     checks and binding
src/codeai/acceptance.py       acceptance and task completion
src/codeai/interpretation.py   versioned interpretation of preserved observations
src/codeai/providers.py        model/provider boundary
experiments/                   preregistered experiments and frozen reports
tests/                         executable invariants and regressions

The repository is:

https://github.com/ernanhughes/codeai

The book’s preserved capstone evidence also lives with the manuscript under:

experiments/applied-ai/evidence/

That separation is intentional: implementation, historical evidence and explanatory prose are related, but they are not the same artifact.

Use it

Clone CodeAI and run its tests:

git clone https://github.com/ernanhughes/codeai.git
cd codeai
python -m pip install -e ".[dev]"
pytest

Create a durable run:

codeai run create "Investigate the failing repository test"
codeai run list
codeai run show <run-id>

Configure logical models and inspect them:

codeai models init
codeai models list

Run model experiments only when you have a question worth measuring. CodeAI’s experimental path can compare repeated sampling from one model with heterogeneous sampling under matched task and verification conditions rather than assuming that more models automatically means more useful diversity.

Operating boundary

CodeAI is an experimental runtime and evidence vehicle, not a claim of production readiness.

It demonstrates narrower properties that can be inspected:

• append-only durable process state;
• explicit task, call and attempt identity;
• deterministic context compilation;
• preserved observations before interpretation;
• authority checks around effects;
• idempotency and replay behavior;
• verification bound to artifacts or observed state;
• acceptance as a distinct operation;
• deterministic next-operation policy;
• explicit unresolved outcomes;
• replayable evidence for experiments and audits.

It does not thereby establish:

• authenticated real-world identity;
• distributed concurrency correctness;
• containment of arbitrary execution adapters;
• universally adequate verifiers;
• general causal proof that an action produced every observed change;
• safe autonomous operation;
• that one routing or model-selection policy is globally optimal.

Those are different claims and require different evidence.

What comes next

The important direction is not adding autonomy for its own sake.

It is making more of the process explicit while preserving the boundaries that already work:

• richer control surfaces over the same durable runtime;
• stronger recovery and reconciliation for uncertain external effects;
• more realistic multi-task and concurrent execution;
• calibrated routing and escalation only after measured promotion;
• better views over claims, evidence, decisions, costs and unresolved work;
• more capstone workloads that test composition rather than isolated components;
• keeping the book, implementation and preserved evidence synchronized.

The destination is a practical rule for applied AI:

use deterministic machinery where the rule is known
spend model intelligence where cognition is useful
preserve what happened
verify what matters
authorize effects explicitly
and make the next operation a software decision

CodeAI is that rule made inspectable.