This book is about building systems around models. Before we build planners, memory, tool routing, critics, search and verifiers, we need an answer to a question that turns out to be harder than it looks:

What is an agent?

The word is currently applied to almost everything: a single LLM call, a chatbot, a fixed pipeline, a tool-using loop, and any five model calls with class names ending in Agent. Those systems may all be useful. But when one word covers all of them, it stops telling us anything about the computation, and we lose the ability to say which mechanism is doing the work.

So this chapter builds the smallest set of distinctions that survive contact with real systems. We will move through six steps, each one adding a property the previous step lacked: a model call, a fixed workflow, model output used as control, an action that changes an environment, a result that returns as an observation, and finally an observation that changes the next decision.

The last step is the one that matters, and it gives us the sentence the rest of the book rests on:

A model produces an output. An agent uses state and observations to decide what happens next.

1. Start with one model call

The smallest possible LLM program has no architecture at all:

answer = llm("Explain gradient descent")
print(answer)

The program supplies an input, the model returns an output, and the program ends. For a great many tasks that is exactly the right shape. Rewriting a paragraph, summarising a document, classifying a message, extracting fields, translating text — for all of these, one model call is cheaper, faster and easier to test than anything with a loop in it.

Calling that program an agent adds nothing to our understanding of it. It is a model invocation.

This distinction earns its place early because one of the most common failures in agent engineering is architectural rather than technical: adding a runtime, memory, tools and loops to a task that never needed any of them. So we will carry one rule from here to the last page.

Do not add an agent mechanism unless it solves a failure you can identify.

2. Software structure is not agentic control

Wrap that call in a class and the picture changes less than it appears to:

class Assistant:
    def __init__(self, model):
        self.model = model

    def run(self, task):
        prompt = f"Solve this task:\n\n{task}"
        return self.model(prompt)

We have gained real software structure — configuration, prompt construction, a model client, logging, retries, caching, telemetry. None of it touches the control flow, which still runs from task to prompt to model to answer along a route fixed before execution started. Naming the class Assistant, Worker or Agent does not change the computation.

Now scale that up. Consider a system with five distinct model calls and five distinct prompts:

research = researcher(task)
plan     = planner(task, research)
draft    = writer(task, plan)
review   = critic(draft)
final    = reviser(draft, review)

The classes might well be called ResearchAgent, PlannerAgent, WriterAgent, CriticAgent and RevisionAgent. Some frameworks would describe this as a multi-agent system. Structurally, it is a chain:

    flowchart LR
    R[research] --> P[plan] --> W[write] --> C[review] --> V[revise]
  

The programmer specified that control graph before execution began. A workflow may still contain branches, retries and conditions, but those transitions are part of the predefined program structure. That makes it a workflow, and this is not a criticism — a workflow is frequently the better design because it is easier to reproduce, test, budget, secure and reason about. There is no prize for making software less deterministic. So instead of arguing about the label, ask a more useful question:

Who decides what happens next?

If the next transition follows a control structure defined in advance, we have a workflow. If a runtime policy selects the next action from the current state and observations, something genuinely different has been introduced.

3. Control is the dividing line

Suppose a model returns the single token SEARCH, and the program uses it like this:

if decision == "SEARCH":
    observation = search_web(query)
else:
    observation = None

The model output is no longer an answer destined for a user. It is a control signal, and the program has separated three things that are usually fused: deciding what to do, representing that decision, and executing it.

This is the boundary where agentic systems begin. But a single decision does not yet make a loop, because nothing has come back. The missing ingredient is feedback.

4. The smallest useful loop

Add feedback and the whole structure collapses into three lines:

while not done:
    action = decide(state)
    observation = execute(action)
    state = update_state(state, action, observation)

    flowchart TD
    S[state] --> D[decide]
    D -->|action| E[execute]
    E --> V[environment]
    V -->|observation| U[update state]
    U --> S
  

The cycle in that diagram is the entire point. The route through the system now depends on results that did not exist when the run started, which gives us the working definition for this book:

An agent is a stateful system that selects actions, executes them against an environment, observes the results, and uses those observations to influence subsequent decisions.

The definition is deliberately narrow and deliberately unglamorous. It says nothing about consciousness, requires no LLM, assumes no long-term memory and grants no unrestricted autonomy. It requires exactly one thing: a feedback loop in which observations can alter future action selection.

None of this is new. The agent-environment loop long predates LLM agents, and reinforcement-learning texts use the same separation between agent, action, environment and resulting experience.[1] What ReAct and its descendants did was place a language model inside that older interactive structure, interleaving model reasoning with actions and the observations they return.[2]

5. Three things, named precisely

We now have enough to be exact about the taxonomy, which will save a great deal of argument later.

Real systems combine all three, and usually should: fixed preprocessing, then an agent loop, then fixed verification. That hybrid shape is common enough that arguing about whether it “really counts” as an agent is wasted effort. The taxonomy is a tool for locating where adaptive control lives, not a category to defend.

6. The environment is whatever can answer back

Environment sounds more exotic than it is. For a robot it may be the physical world; for everything else it is simply the set of things the system can act on and receive information from.

What these have in common is not physicality. It is that the agent can obtain information from them, act on them, or both — which means the system can receive new evidence during execution.

That is the property that makes a loop worth building. Where no meaningful new information can arrive, a loop has nothing to learn from and is just an expensive way to call a model repeatedly.

7. Build the smallest agent we can

Here is an intentionally tiny agent with an action space of exactly two entries: CALCULATE and FINAL.

The code below is a pedagogical sketch of the control loop. The runnable implementation in the companion repository uses the same structure with typed actions, observations and explicit transition records. Its acceptance boundary deliberately accepts already-typed actions, so that this chapter can isolate a single question: can an observation change the next action? The next chapter replaces those typed actions with untrusted proposals that have to cross a real boundary.

State first, because everything else is defined against it:

from dataclasses import dataclass, field


@dataclass
class AgentState:
    task: str
    history: list[dict] = field(default_factory=list)
    done: bool = False
    final_answer: str | None = None


@dataclass
class Action:
    name: str
    argument: str

The model proposes the next action:

def decide(model, state: AgentState) -> Action:
    prompt = f"""
Task:
{state.task}

History:
{state.history}

Choose exactly one action:

CALCULATE <expression>
FINAL <answer>
"""
    output = model(prompt).strip()
    name, argument = output.split(" ", 1)
    return Action(name=name, argument=argument)

The runtime executes it:

def execute(action: Action):
    if action.name == "CALCULATE":
        return {"type": "calculation", "value": safe_calculate(action.argument)}

    if action.name == "FINAL":
        return {"type": "final", "value": action.argument}

    return {"type": "error", "value": f"Unknown action: {action.name}"}

And the loop closes over both:

def run_agent(model, task, max_steps=5):
    state = AgentState(task=task)

    for step in range(max_steps):
        action = decide(model, state)
        observation = execute(action)

        state.history.append({
            "step": step,
            "action": action,
            "observation": observation,
        })

        if observation["type"] == "final":
            state.done = True
            state.final_answer = observation["value"]
            break

    return state

The model did not change. The computation around it did, and that is the central move of agent engineering.

The companion experiment for this chapter makes the causal claim testable rather than rhetorical. Its first action is RUN_TESTS. If the returned observation is a failure, the next action becomes READ_FILE; if the tests pass, the agent stops instead. The test asserts both trajectories directly, so a single observation demonstrably changes what the same policy does next.

8. The most important boundary: proposal is not execution

The code above is far too trusting. The model returns CALCULATE 19 * 23 and the runtime obliges — but the same model could just as easily return DELETE_DATABASE production, or malformed text, or a tool name that does not exist. Nothing in that loop distinguishes a reasonable proposal from a catastrophic one.

The architecture we actually want inserts a decision point between the two:

    flowchart LR
    M[model] -->|proposed action| R
    subgraph R[runtime]
        direction TB
        P[parse] --> V[validate] --> A[authorize]
    end
    R -->|accepted action| X[executor]
    X --> E[environment]
    E -->|observation| M
  

The model proposes. The runtime decides what is allowed to become real. The environment reports what actually happened. That separation is load-bearing enough to state as a rule:

The model proposes. The runtime decides what may execute. The environment provides evidence about what actually happened.

Research on tool-using models approaches the same separation from the other side. Toolformer studies a model learning when to call an external API, which one to call, and how to use the result.[3] Our engineering contribution is to wrap those probabilistic decisions in a deterministic boundary — which is what the next chapter builds.

9. State is runtime truth, and history matters only when it changes a decision

The loop needs information that survives from one step to the next. That is state:

state = {
    "task": task,
    "step": 3,
    "last_action": "run_tests",
    "last_observation": "2 tests failed",
    "files_changed": ["src/parser.py"],
}

This is a different thing from asking the model to reconstruct the situation from a conversation transcript. If something matters enough to control execution, represent it explicitly rather than hoping it survives summarisation.

It is worth drawing one line now that the memory chapter will need later. State is the runtime’s current explicit representation of what matters for this execution; memory is selected past information that may be retrieved to affect a later decision. State can still be incomplete or stale. That is why later observations and verification remain necessary. A lesson learned on a previous task belongs in memory. Conflating them causes real bugs, because they have different lifetimes and different failure modes.

Now consider what a trajectory looks like once state accumulates:

Each observation changes what should sensibly happen next. Without state the system can only ever ask what should I do? With a trajectory it can ask the far more useful question: given what has happened, what should I do now?

This is why storing a transcript is not the point. The property we want is history dependence — past events mattering because they change current state, available actions, confidence, progress or stopping conditions.

10. Failure is useful once it becomes an observation

Suppose the agent tries READ_FILE config.yaml and the environment returns:

{ "ok": false, "error": "FileNotFoundError" }

In a one-shot call that is the end of the story: the answer was wrong and the program finished. In a loop it is new information. A later policy can choose LIST_DIRECTORY, discover config/settings.yaml, and recover.

The loop earns its complexity when interaction produces or reveals information during execution that should change a later decision. This is why ReAct is a useful reference point rather than merely a prompting trick: its contribution is the interleaving of model-generated reasoning and actions with observations returned by a real external environment.[2]

11. The policy does not have to be a model

The decide() function is the policy — the mapping from state to action. An LLM is one implementation of it. So is this:

def decide(state):
    if state["tests_failed"] > 0:
        return Action("INSPECT_FAILURE", "")

    if not state["tests_run"]:
        return Action("RUN_TESTS", "")

    return Action("FINAL", "done")

So are rules, classifiers, small neural models, frontier LLMs, search algorithms, and mixtures of several of these. Which means the decomposition we want is not agent = LLM + prompt. It is a runtime, plus state, plus a policy, plus an action space, plus environment feedback — with the policy as one replaceable component among five.

Seeing it that way immediately suggests an economy. When state says the phase is verification and the only legal action is RUN_TESTS, there is nothing for a large model to decide. The program already knows:

if state["phase"] == "verification":
    return Action("RUN_TESTS", {})

Model judgment should be reserved for decisions where ambiguity genuinely exists, which gives us a second rule:

Do not delegate a decision to the model merely because the model can make it.

The strongest systems tend to alternate between the two modes rather than committing to either:

    flowchart TD
    A[validate input] --> B[choose investigation action]
    B --> C[execute tool]
    C --> D[validate observation]
    D --> E[choose next investigation]
    E --> F[run final checks]

    classDef det fill:#e8e8e8,stroke:#666,color:#000
    classDef ada fill:#fff3cd,stroke:#b8860b,color:#000
    class A,C,D,F det
    class B,E ada
  

Grey steps are deterministic; amber steps are adaptive. Agency is not maximal model control. It is adaptive control placed where adaptive control is actually useful.

12. The action space is the practical capability

A model may be able to discuss thousands of tasks. An agent can only change its environment through the actions the runtime exposes, so the action space — not the model — sets the real capability ceiling:

ACTIONS = {
    "SEARCH": search,
    "READ_FILE": read_file,
    "PROPOSE_PATCH": propose_patch,
    "RUN_TEST": run_test,
    "FINAL": finish,
}

Compare a single RUN_SHELL(anything) against READ_FILE(path), PROPOSE_PATCH(diff) and RUN_TEST(test_name). The first is broad and semantically coarse. It can still be logged and sandboxed, but it gives the runtime a less precise surface for validation, authorization and auditing. The narrower actions make the intended capability explicit, which makes each attempt easier to reason about and constrain. The second constrains what can be attempted, and in exchange makes each attempt legible.

That trade runs in both directions and there is no free position on it. A broader action space buys flexibility at the cost of controllability; a narrower one buys controllability at the cost of flexibility. The goal is not maximum autonomy but the smallest action space that still solves the task.

13. When the loop earns its cost

The condition is narrow and worth memorising:

Use an agent loop when information arrives during execution that should change future actions.

That property is real in more tasks than it might seem. Debugging produces an actual failure message that determines what to inspect next. Research surfaces evidence that reveals what is still missing. Repository work produces validation failures that redirect the next edit. Investigation collects results that revise the hypothesis and therefore the next experiment. In each case the task unfolds, and the unfolding carries information.

The inverse condition is just as important:

If every job genuinely follows the route transcribe → extract entities → summarise → format report, then hard-code that route. An agent would only add more possible trajectories without adding capability.

The underlying trade-off is worth stating plainly, because it is easy to talk about autonomy as though it were free intelligence. More autonomy means more possible trajectories, which means more flexibility — and simultaneously more failure modes, harder reproduction and more variable cost. Autonomy is an architectural choice with a bill attached.

14. What the rest of the book adds

We now have the primitive. Every remaining chapter adds exactly one mechanism to it, and each one is introduced because the previous configuration fails in an identifiable way.

Read as a single construction, the arc runs from control through valid action, alternatives, revision, planning, progress, capabilities, memory, search and evidence to integration.

By the last chapter the agent will be substantially more capable, and the model at the centre of it may be exactly the same model we started with. What will have changed is the system computation around it.

15. The book’s engineering rule

One rule carries through every chapter:

Move as much correctness as possible out of probabilistic model behaviour and into explicit, inspectable software.

This is not an argument for removing the model. The model is valuable precisely where fixed rules are hard to enumerate: interpreting messy observations, generating candidate actions, comparing alternatives, forming hypotheses, proposing plans. Those are genuinely difficult to specify in advance, and pretending otherwise produces brittle systems.

The rule is about what happens as a decision approaches a real side effect. At that point the surrounding software should become progressively more concrete — the model proposes, the runtime validates, the executor acts, the environment changes, the runtime observes, and the verifier checks. Each step in that chain is a place where a probabilistic decision is converted into an inspectable one.

This decomposition is not idiosyncratic. The LLM-agent literature keeps arriving at the same recurring components — planning, memory, action and tool use, environment interaction — even where individual papers and frameworks use incompatible terminology for them.[4] Our aim is not to reproduce any one framework, but to understand why these particular pieces keep reappearing.

16. A final test

Suppose an unfamiliar system is described as an “AI agent”. Do not begin by asking which framework it uses. Ask instead:

1. What is the state?
2. What does the model actually decide?
3. What actions can it propose?
4. What software validates those proposals?
5. What actually executes them?
6. What can change in the environment?
7. What observation returns?
8. How can that observation change the next decision?
9. What stops the loop?
10. What evidence proves the goal was achieved?

Answer those ten questions and the word agent stops being mysterious, because you can see the machine underneath it. Each question also names a chapter of this book, which is not a coincidence: the list is the book’s table of contents written as a diagnostic.

Research roots

This book is an engineering reconstruction rather than a literature survey, but the ideas we build have clear research ancestry. References are selective by design — they mark where a mechanism connects to established work, not every paragraph that could carry a citation.

1. Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018. The agent-environment interaction loop provides an older formal vocabulary for actions, observations, state and adaptive decision-making. https://www.incompleteideas.net/book/bookdraft2018mar21.pdf
2. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” ICLR 2023. ReAct explicitly interleaves model reasoning, environment actions and returned observations. https://arxiv.org/abs/2210.03629
3. Timo Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools,” NeurIPS 2023. Toolformer studies language models learning when and how to call external APIs and use their results. https://arxiv.org/abs/2302.04761
4. Lei Wang et al., “A Survey on Large Language Model based Autonomous Agents,” Frontiers of Computer Science 18, 2024. The survey organises LLM-agent systems around recurring components and provides a useful map of the field. https://doi.org/10.1007/s11704-024-40231-1

Next: The Action Boundary

We now have something concrete enough to improve, and the weakest point in it is the place where a proposal becomes a side effect. In our tiny loop, model output travels to execution with nothing in between.

The next chapter replaces that gap with a real boundary: a structured proposal that is parsed, validated and authorised before an executor is allowed to touch anything. It is the first step toward a reliable agent, and it follows from a single commitment:

Never let raw model output become a side effect merely because it looked plausible.