Applied AI

A model gives you intelligence. Applied AI is the engineering required to make that intelligence participate reliably in a process.

Calling a model is easy.

You send it some text. It sends some text back.

The difficult part is everything around that exchange.

What was the model supposed to accomplish? Which information was it allowed to see? Which version of that information did it receive? What happened if the request failed halfway through? Was the answer merely plausible, or was it checked? Was it permitted to change anything? If it changed something, did the change actually happen? Did it produce the result you wanted? What should happen next?

In a chat window, a person quietly answers all of those questions.

They remember the objective and choose the context. They notice when the model misunderstood and decide whether to believe it. They copy the useful part somewhere else, authorize the effect, and decide whether the work is finished.

The chat interface makes the model visible and the process invisible.

This book is about making the process visible.

That does not mean the chat window goes away. You will still type and talk to a model every day; conversation is the most direct way to say what you want. What leaves the chat box is the process around it. And once the process is software rather than habit, it can be shaped around the way you work. The last chapter comes back to that.

The problem is no longer getting intelligence

For most of the history of software, intelligence was the scarce component.

If a task required understanding an unfamiliar document, proposing a repair, comparing several plausible explanations, or producing something genuinely new, a programmer had two choices:

write rules for it

or

give it to a person

Language models added a third.

software
   ↓
model
   ↓
proposal

That is a profound change.

But it is not yet a system.

The model is stochastic. Its output can change when the apparent inputs have not. Its capabilities move from release to release. The same named service can change underneath an application. It can produce convincing errors rather than exceptions. A successful HTTP request says almost nothing about whether the work succeeded.

And the model is only one component in the process. Around it remain questions of state, evidence, cost, authority, side effects, recovery, verification and human judgment.

Those questions become more important as the model gets better, not less.

A weak model forces you to inspect everything, while a strong model can be right forty times in a row and quietly train you not to inspect the forty-first.

The durable question

This book is therefore not primarily about prompting, agents, or whichever model is strongest when you read it.

Those things will change.

The durable question is:

How do you put intelligence inside a process without making the process itself unreliable?

The answer developed through this book is surprisingly conservative.

Most of the system should be ordinary software.

Use deterministic machinery wherever the correct operation can be specified. Use the model only where you genuinely need a proposal from a space you could not enumerate in advance.

Then surround that stochastic boundary with everything ordinary software already knows how to do well:

intent
  ↓
explicit state
  ↓
selected context
  ↓
model proposal
  ↓
preserved observation
  ↓
evidence
  ↓
authority
  ↓
action
  ↓
independent verification
  ↓
new state
  ↓
next operation

The model supplies cognition without owning the process.

One stochastic box

A central claim of this book is that stochasticity should be confined, not spread through the architecture.

Consider an AI-assisted document review.

The system may need to:

• determine whether a paragraph changed;
• retrieve the current version;
• check a budget;
• identify claims that may require evidence;
• locate the flagged spans;
• verify that cited sources exist;
• decide whether an edit is authorized;
• write a change;
• run checks against the result;
• record what happened.

Only one of those operations clearly benefits from a model: identifying claims whose need for evidence cannot be completely specified in advance.

Everything else has a right answer.

So the rule used throughout this book is:

A component belongs on the deterministic side unless it demonstrably cannot be.

There is an even simpler test:

If you would be annoyed to get a different answer when you rerun it, it probably should not be a model call.

Stochasticity is not something we are trying to eliminate; it is the reason the model is useful.

The model can propose the interpretation you did not anticipate, the repair you did not enumerate, or the connection you had not considered.

The mistake is paying for that variance in places where you wanted certainty.

Verification changes everything

There is an asymmetry underneath almost every successful AI application.

Producing an answer can be difficult.

Checking an answer is often much easier.

You may not know how to generate the correct code, but you can run the tests.

You may not know which citation belongs in a paragraph, but you can check whether the proposed source actually contains the claim.

You may not know the right patch, but you can compile the result.

This is why software was such fertile ground for AI. It was not because programming was easy; software engineering had spent decades constructing unusually cheap verifiers: compilers, type systems, tests, continuous integration, version control and reversible changes.

The model arrived in terrain that had already been mapped.

That gives us a useful way to evaluate other domains.

Text plus intelligence can produce an impressive demo.

Text plus intelligence plus a cheap verifier can produce a dependable process.

Where the verifier does not yet exist, building it may be more important than choosing a better model.

The human does not disappear

The architecture in this book deliberately leaves four jobs outside the model.

Intent — deciding what should happen and what would count as done.

Authority — deciding which effects are permitted.

Verification — obtaining evidence about what actually happened.

Frontier judgment — deciding whether a model is the right mechanism for this operation at all.

These are not leftovers waiting for the next model generation to absorb them; they are the jobs whose importance grows as generation becomes cheaper.

If a model can produce one hundred times as much work, the system needs more ability to decide what work should exist, more ability to control what may happen, and more ability to tell whether any of it was correct.

That is why this book treats human review as an architectural component rather than a final checkbox.

A person staring at hundreds of mostly-correct outputs eventually becomes a rubber stamp. Decades of automation research suggest that diligence alone does not solve this.

So the process has to make good review cheap:

mechanical checks first
        ↓
evidence attached to claims
        ↓
attention routed by risk
        ↓
human judgment where judgment is actually required

Human-in-the-loop is not a safety mechanism merely because a human appears somewhere in the diagram. The loop has to contain information the human can realistically evaluate.

The book is also a build

The reference system is CodeAI.

We begin with almost nothing: a callable model.

Then we add one engineering obligation at a time.

The model call acquires identity. Attempts become distinct from logical calls. Raw responses survive after the process exits.

Different provider protocols are normalized without pretending their differences do not exist. Context becomes an explicit input instead of whatever happens to be in a transcript, and history becomes durable state.

Claims become separate from the evidence supporting them. A proposed action becomes separate from an authorized action, and an action becomes separate from its observed effect.

Verification becomes independent of the component that generated the proposal. Retries become part of the effect model rather than a convenience hidden inside an HTTP client. Several proposals can be isolated from one another.

Experiments can measure whether more calls, different models, or different prompts actually buy anything. Finally the runtime can answer a question more important than which model should I call?

It can answer:

What operation should happen next?

Sometimes the answer is CALL. Sometimes it is CHECK or ACTION. Sometimes ASK_HUMAN. Sometimes the correct answer is STOP.

Choosing the operation comes before choosing the model.

One runtime, many windows

The process also cannot live inside a chat interface.

If your phone remembers one conversation, your editor another, your terminal a third and your browser a fourth, you do not have one intelligent system.

You have four isolated processes and a person manually synchronizing them.

The architecture developed here reverses that relationship:

             phone
               │
editor ──── runtime ──── browser
               │
            terminal

The surfaces become views.

The durable runtime becomes the source of truth.

That leads to another distinction that runs through the book:

Memory is durable. Context is selected.

Remembering everything does not mean sending everything to every model call.

The system may retain a rich history while deliberately selecting only the information relevant to the current operation.

The important difference is that exclusion becomes a recorded decision rather than accidental amnesia.

The model is not the foundation

There is another reason to separate the runtime from the model.

The model will change.

The strongest model available next year will not be the strongest one available today. Prices will change. Providers will disappear. Models will regress on individual tasks while improving overall. A prompt tuned for one model may behave differently on another.

So the system should not be built on a model. It should have places where models can be installed.

The book calls these chambers.

fast-classify  → current occupant
draft          → current occupant
deep-review    → current occupant
critic-a       → current occupant
critic-b       → current occupant

The chamber is named for the job.

The model occupying it is temporary.

This gives us a different approach to upgrades:

A new model release is an experiment, not an upgrade.

Replay the work that matters. Compare the candidate with the incumbent per item.

Count the cases that improved and those that regressed, then measure the cost per accepted result.

Change the occupant only where the evidence justifies it.

Finish the frame; turn the chamber.

More intelligence is not automatically better

This book does not stop at proposing that architecture.

It tests one of its most tempting assumptions.

If one model call is useful, surely several are better. If several draws from one model help, surely several different models help more. If different models fail differently, surely diversity buys coverage.

Reasonable ideas, but exactly the kind experiments are for.

A frozen series of experiments in the later book compares repeated sampling, heterogeneous model portfolios, harder tasks, prompt-stance diversity and a preregistered replication.

Some of the attractive results disappear. Some become weaker when the tasks become harder. One promising subgroup survives long enough to justify another experiment and then fails to replicate.

That is not a detour from the book’s argument.

It is the argument.

Discovery is not promotion. A signal is not a result. One successful run is not a measurement.

The runtime should spend intelligence where intelligence has demonstrated value, not where adding another model call merely feels sophisticated.

Intelligence has a price

There is also an economic reason to care about the boundary.

Traditional software has an attractive shape:

expensive to build
       ↓
cheap to run

Put a frontier model permanently into the request path and part of that relationship reverses.

Every user action costs money again — retries, evaluations, and every unnecessary piece of context.

Runtime intelligence is rented capability.

So the book develops a cost ratchet:

remove unnecessary context
        ↓
use the cheapest model that passes
        ↓
escalate failures
        ↓
route by task
        ↓
distil narrow capability where appropriate
        ↓
replace the operation with deterministic software

Where a verifier exists, intelligence has a ceiling.

Once the cheaper system passes the required check, spending more does not make the result more passed.

Where there is no verifier, spending has no natural stopping point — and that is precisely where it becomes easiest to convince yourself that the more expensive model must be better.

Measurement is therefore part of the cost architecture, not a separate concern.

Build the measurement first

Before the runtime becomes sophisticated, we build the thing that can tell us whether sophistication helped.

A small frozen task set. Mechanical checks. Repeated runs. Recorded model and prompt identities. Error bars rather than anecdotes.

Then every later claim has somewhere to land.

Did the new model improve the task? Did another sample add coverage? Did the critic agree with the actual verifier?

Did the prompt intervention survive another set of draws? Did the system become cheaper per successful outcome? Did the project actually finish faster?

Without measurement, model selection becomes reputation, prompting becomes folklore, and architecture becomes taste.

With measurement, each becomes an engineering decision that can be reversed when the evidence changes.

What you should be able to do after reading this book

The goal is not that you remember thirty chapter titles.

The goal is that you can look at an AI-enabled system and ask better questions.

You should be able to identify:

• which operation genuinely requires a model;
• which operations should be deterministic;
• what exact task a call belongs to;
• what context the model was permitted to see;
• what it actually received;
• which provider and model produced an observation;
• how many attempts occurred;
• which bytes were returned before anyone interpreted them;
• what the model merely asserted;
• what evidence supports those assertions;
• whether an action is available versus authorized;
• what effect actually happened;
• whether verification was independent of generation;
• what state can safely be resumed after a crash;
• whether a retry can duplicate an effect;
• whether another model call would add useful information;
• whether a more expensive model earns its price;
• whether a promising result survived replication;
• what the process should do next;
• and whether it can stop honestly when the objective has not been established.

Most importantly, you should be able to distinguish five roles that are routinely collapsed:

model       → cognition

runtime     → coordination

tools       → action

verifiers   → evidence

human       → intent + authority

That separation is the architecture.

The journey through the book

The six parts follow the construction of that architecture.

Part I — Where You Stand

We begin before the implementation.

What work should remain deterministic? Why did software automate before many apparently easier domains? Why does human review degrade when the AI becomes reliable? What does intelligence actually cost? How do you measure a stochastic system? And if AI is making programmers dramatically faster, why are projects not finishing dramatically faster?

The conclusion changes the target.

Generation is not the primary bottleneck.

Intent, authority and verification are.

Those become the engineering problem for the rest of the book.

Part II — Get the Model Out of the Chat Box

Now we build the boundary.

The interface stops owning the process. Model identity becomes replaceable. Calls, attempts and tasks acquire durable identities. Several provider protocols are made usable without pretending they are identical. Completion becomes something explicitly established rather than inferred from a successful generation.

By the end of this part, a model is no longer a conversation partner.

It is a component.

Part III — Give Intelligence a Runtime

The component acquires a world around it.

Context becomes explicit and reproducible. Working state survives a restart. Raw observations are preserved before interpretation. Claims point back to evidence. Decisions can explain what they relied on. Proposed actions become observable effects.

This is where AI work stops disappearing into scrollback.

Part IV — Make It Safe and Verifiable

Capability and authority separate.

The fact that a system can perform an action no longer means it may perform it.

Verification is bound to the actual resulting state rather than to the model’s description of that state. Retries are treated as potential effects. Idempotency, stale state and crash windows become explicit engineering problems.

The system learns to say more than “success.” It learns what kind of success has actually been established.

Part V — More Intelligence Is Not Automatically Better

With the machinery built, we use it to test ourselves.

Independent proposals, different models, harder tasks, prompt diversity and replication become controlled experiments rather than intuitions.

The point is not to discover one eternal model strategy.

The point is to build a process capable of discovering when a strategy stops earning its cost.

Part VI — Put Intelligence Into the Process

Finally the pieces compose.

The runtime looks at explicit state and decides which kind of operation is required next.

Not which model sounds cleverest.

Which operation.

The capstone runs one task end to end through intent, context, generation, preserved observation, evidence, authority, action, verification, acceptance and replay.

Then the process is reopened from its durable record and asked to explain itself.

That is the test.

The book then closes on what the whole construction means for the reader. When software can be built around one person, the process you have learned to build becomes your own application of AI: conversation for intent, tools shaped around your work, a runtime you own, and a model you can replace.

The larger idea

AI is often presented as a replacement for software.

This book reaches almost the opposite conclusion.

The more capable the model becomes, the more valuable ordinary software engineering becomes around it.

State matters because the model has none you can safely assume.

Schemas matter because free-form boundaries spread uncertainty.

Evidence matters because plausible language is not proof.

Authorization matters because capability is cheap.

Verification matters because generation is abundant.

Persistence matters because an observation may need to be reinterpreted long after the call that produced it.

Measurement matters because a stronger model can be worse on the one case you depended on.

And deterministic control matters because intelligence is most useful when it does not have to govern itself.

The result is not an autonomous blob sitting at the centre of the architecture.

It looks much more like ordinary software with a carefully chosen intelligent boundary.

    flowchart TD
    A["deterministic frame<br/>context · routing · bookkeeping"] --> B(["stochastic proposal<br/>model"])
    B --> C["evidence"]
    C --> D["authority"]
    D --> E["action"]
    E --> F["verification"]
    F --> G["durable state"]
  

The model is powerful.

The system is dependable because the rest of the architecture does not require it to be something it is not.

That is Applied AI.

And because that architecture is ordinary software, it does not have to be built for everyone. As construction gets cheaper, the frame, the tools and the policies can be shaped around one person’s work while the model inside stays replaceable. That is your applied AI.

Experimental status

This is a book about a moving technology, so observations and architectural claims are kept separate.

Specific model behavior, provider protocols, prices and experimental measurements belong to the environment and date in which they were observed. Preserved evidence remains preserved when later models or APIs change.

The architectural contracts are the more durable layer:

• explicit state;
• bounded stochasticity;
• preserved observations;
• independent evidence;
• capability separated from authority;
• effects separated from claims about effects;
• deterministic next-step policy;
• model substitutability;
• measurement before promotion.

All thirty chapters are complete drafts across six parts; the last is an argument chapter rather than a construction stage. The experimental arc, runtime arc and end-to-end capstone are drafted, with preserved run evidence retained separately from the manuscript.

Where an experiment produced a negative result, the negative result remains.

Where an implementation cannot establish something, the book says so.

Where the technology changes, a later edition should change with it.

Applied AI begins where the chat box ends.

Continue with Beyond the Chat Box.