The Architecture of Agent Behavior

Chapters 1 and 2 introduced the working style of agent systems: conversation, variation, feedback, revision, and human direction. Now we need to look underneath that experience.

An agent does not become capable because a class is named Agent or because a prompt says “you are a researcher.” The behavior comes from the arrangement of parts around the model:

goal
  ↓
role + instructions
  ↓
context + state
  ↓
model proposes
  ↓
runtime validates
  ↓
tool / memory / message
  ↓
environment observation
  ↓
state update
  ↓
continue, revise, coordinate, or stop

That is the architecture of agent behavior. The model supplies interpretation and generation. The prompt and context shape what the model sees. The runtime owns state, tool execution, permissions, memory retrieval, orchestration, and stopping rules. The human operator supplies purpose, constraints, judgment, and responsibility.

This chapter is about the mechanisms that make an agent behave differently from a plain model call.

Structure Shapes Behavior

A model call produces one response from one context. An agentic system can change what happens next based on what has already happened.

That change can come from many places:

None of these mechanisms is magic. Each is a design choice. The more clearly you can name the mechanism, the easier the system becomes to improve.

Roles Shape the Model’s Frame

Roles are one of the easiest ways to change agent behavior, but the mechanism is often misunderstood.

Giving a component a name like ResearcherAgent does not create a new kind of intelligence. A role usually changes some combination of:

agent role
  =
instructions
+ context
+ available tools
+ memory access
+ allowed actions
+ output schema
+ evaluation criteria

A researcher role may see source material, use search tools, and produce evidence summaries. A critic role may see a draft and a rubric, but not mutation tools. An executor role may receive an approved plan and operate through tightly scoped tools. A manager role may route tasks and compare outputs.

Roles help because they narrow attention. A general assistant may try to plan, research, write, judge, and execute in the same breath. A role asks for one kind of contribution.

Roles also make collaboration possible. If one component gathers evidence and another judges a draft against that evidence, the handoff can be inspected. If both components are just “the agent,” the failure is harder to locate.

The goal is not theatre. It is separation of responsibility.

Tools Extend Action and Observation

A language model by itself does not browse the web, execute code, charge a credit card, update a database, or inspect a private file system. It produces text or structured output. The surrounding runtime may interpret part of that output as a proposed tool call.

A tool-using agent system usually follows this pattern:

model proposes action
        ↓
runtime validates request
        ↓
tool executes
        ↓
environment changes or returns data
        ↓
observation enters state/context

This distinction is critical. The model should not be imagined as directly touching the world. The runtime exposes a controlled action space, checks the request, executes the tool, records the result, and decides what observation returns to the model.

That is the action boundary. The model may propose send_email, run_query, or edit_file, but proposal is not permission and permission is not execution. The runtime can reject a malformed request, ask for human approval, narrow the arguments, run the action in a sandbox, or return an observation that says why the action was not allowed.

Tools expand two things:

Tool design is therefore capability design. A broad shell tool gives enormous power and risk. A narrow lookup_invoice(invoice_id) tool gives less flexibility but more safety. A good architecture chooses tool boundaries deliberately.

Tool results also become evidence. If a code runner reports that tests failed, that observation should be stored as state. The next model call should not merely remember that “something went wrong”; it should receive a structured observation it can act on.

observation:
  tool: test_runner
  command: pytest tests/test_parser.py
  status: failed
  failure: expected ['A', 'B'] but got ['A|B']

Once observations are explicit, the system can revise, retry, escalate, or stop for a reason.

Memory Is Retrieved State, Not Changed Weights

Memory is another place where loose language causes confusion.

Most agent applications do not change the model’s weights when they “remember” something. They store information outside the model and retrieve selected pieces into the model’s context later.

Useful memory design separates several ideas:

Human memory analogies can help, but only as analogies. Software memory is stored, indexed, summarized, embedded, retrieved, expired, and sometimes wrong. It can be stale. It can conflict with current instructions. It can reveal private information if access boundaries are weak.

Memory improves continuity when it is selective. More memory is not automatically better. If the runtime retrieves irrelevant or outdated information, the model may treat it as important simply because it appears in context.

A useful memory system therefore needs more than storage. It needs an eligibility rule for what can be saved, provenance for where it came from, a supersession rule for what replaces old information, and a conflict policy for what happens when memory disagrees with the current task. Without those controls, memory becomes another unreviewed prompt injection surface.

The architectural question is not “does the agent remember?” It is:

What was stored?
Who stored it?
Why is it relevant now?
What retrieved it?
Can the user inspect or delete it?
Should it override the current instruction?

Memory turns isolated interactions into an ongoing process, but only if the system can control what continuity means.

Agents Decompose Work Into Inspectable Steps

Agent systems often perform better when work is broken into steps. The important point is not to expose hidden chain-of-thought. The useful mechanism is to externalize intermediate decisions into inspectable state.

Instead of asking a model to “analyze this data and write a report” in one pass, the system can represent the work as:

1. identify the question
2. inspect the available data
3. extract candidate findings
4. choose the strongest findings
5. draft the report
6. critique the draft
7. revise or stop

Each step can have inputs, outputs, tools, and acceptance criteria. The model may propose a step, but the runtime can store it as a plan. The model may suggest an action, but the runtime can validate it. The model may summarize an observation, but the original observation can remain available for verification.

This is where agent behavior starts to differ sharply from a single response. The system has intermediate artifacts:

plan
checklist
tool call
observation
draft
critique
revision
decision

Those artifacts let a human or another agent inspect the process. They also give the system places to recover. If the draft is weak, the system can return to the outline. If a tool call fails, the system can choose another route. If the evidence is insufficient, the system can ask for more information.

Structured steps are not always needed. For a simple summary, they may add cost and friction. They earn their place when the task is ambiguous, multi-stage, risky, or worth auditing.

Reflection Needs a Judge

Reflection is often described as an agent criticizing and improving its own work. That is close, but incomplete.

A stronger reflection loop looks like this:

generation
    ↓
critique
    ↓
revision proposal
    ↓
comparison / validation
    ↓
accept or reject

The final step matters. A critic can be wrong. A revision can be smoother but less accurate. A model can produce a convincing explanation for a bad change.

Reflection becomes useful when the system has criteria for deciding whether the revision is better. The criteria might be human judgment, tests, a rubric, a deterministic validator, a second model acting under a different role, or a comparison against source evidence.

For example, a writing agent might produce a paragraph. A critic role might say it is vague. A reviser might make it sharper. But the system should still ask:

Did the revision preserve the original claim?
Did it remove evidence?
Did it introduce unsupported certainty?
Is it actually clearer to the target reader?

Reflection is not automatic self-improvement. It is a mechanism for generating candidate improvements. Evaluation decides what deserves to replace the current version.

This prepares the ground for Chapter 5, where reflection, comparison, regression, and versioning become the central subject.

Coordination Turns Roles Into Systems

Multiple agents are not automatically better than one. They add latency, cost, handoff errors, duplicated work, inconsistent state, and role confusion.

They help when specialization pays for that overhead.

Specialization can help because different roles can have:

different context
different tools
different memory access
different output formats
different evaluation criteria
different authority

A researcher can gather evidence without being allowed to edit the final document. A critic can inspect a patch without being allowed to apply it. An executor can run an approved action without owning the goal. A reviewer can decide whether the result meets the standard.

Coordination is the architecture that connects those responsibilities.

manager
  ↓
researcher -> evidence
  ↓
writer -> candidate
  ↓
critic -> weaknesses
  ↓
reviser -> updated candidate
  ↓
reviewer -> accept / reject

Coordination can happen through messages, shared state, task queues, memory stores, event logs, or explicit workflow steps. The more complex the system becomes, the more important the handoff format becomes.

An unstructured handoff says:

Here is what I found.

A structured handoff says:

role: researcher
claim: "Policy changed in version 2.1"
evidence: source URL or document path
confidence: medium
open_questions:
  - source date needs verification

The second is easier for another agent or a human to judge.

Chapter 6 will go deeper into centralized, hybrid, and decentralized architectures. For now, the principle is enough: coordination is valuable when it makes responsibility clearer, not when it merely adds more voices.

Emergent Behavior Through Interaction

Emergence does not need mystical language.

System-level behavior can arise from interactions between components even when no single component explicitly contains the complete behavior.

For example:

The system appears more adaptive than any one call because information moves through the loop. A critic’s objection changes the reviser’s context. A failed test changes the planner’s next option. A memory entry changes what a researcher retrieves. A policy threshold changes whether the system acts or asks for help.

That is emergence in agent architecture: behavior produced by structured interaction.

Emergence Example

Try this in a chat interface:

You are coordinating three roles:
- Manager: breaks the task into steps.
- Researcher: gathers considerations and tradeoffs.
- Critic: identifies weak reasoning and missing constraints.

Use a visible shared memory section with:
- current goal
- known facts
- open questions
- next action

Task: evaluate whether to buy a new laptop.
After each role contributes, update shared memory and choose the next role.

This is not a production multi-agent system. It is a simulation inside one conversation. Still, it makes the mechanism visible. The “team” behavior comes from role separation, shared state, critique, and routing.

Step 1: Manager to Researcher

Manager

Identify the important criteria for deciding whether to replace the current laptop. Include technical, financial, and usage considerations.

Researcher

Useful criteria include:

Technical

• current CPU, memory, storage, GPU, and battery capability;
• recurring performance problems;
• software and workload requirements;
• expected useful life of the current machine.

Financial

• purchase budget;
• resale value of the current machine;
• repair or upgrade options;
• alternative uses for the money.

Usage

• daily workloads;
• mobility requirements;
• upcoming work that may need additional performance;
• tolerance for setup and migration time.

The result is recorded in shared state.

Step 2: Manager to Critic

Manager

Review the proposed criteria. Identify anything important that is missing, irrelevant, or potentially biased.

Critic

The criteria are useful, but the decision could be improved by adding:

• opportunity cost of waiting;
• repairability and environmental considerations;
• whether the current problems are caused by hardware limits or software configuration;
• risk that a new purchase solves annoyance rather than a real constraint.

The Manager can now decide whether those suggestions justify revising the decision criteria.

    flowchart LR
    M[Manager]
    R[Researcher]
    C[Critic]
    S[(Shared state)]

    M -->|Research task| R
    R -->|Findings| S
    M -->|Review task| C
    S --> C
    C -->|Critique| S
    S --> M
  

The result may feel like a small organization. Architecturally, it is a structured conversation with roles, shared state, critique, and routing.

Adaptation and Evolution

The older language of agents often says “the agent learns.” Sometimes that is a useful shorthand. Architecturally, it hides three different mechanisms.

Runtime Adaptation

The system changes behavior because the current situation changed:

context changed
state changed
tool returned new information
memory retrieval changed
budget changed
user corrected the direction

No model weights need to change. The next output differs because the input environment differs.

System Improvement

Developers, users, or automated optimization processes change the system:

prompt
policy
retrieval rule
tool schema
model choice
routing logic
evaluation criteria
memory policy

This is closer to software evolution. The agent system becomes a new version of itself because some part of the architecture changed.

Model Training

The model itself changes only when its parameters are updated through a training process such as fine-tuning, preference optimization, reinforcement learning, or another provider-side training method.

That is a different kind of change and should not be confused with memory or prompt updates.

These distinctions protect you from false confidence. If a system performed better today, you want to know why. Did it retrieve a better memory? Did a tool return different data? Did the prompt change? Did the provider upgrade the model? Did the user give clearer feedback?

Without versioning, all of those explanations collapse into one vague story: the agent improved.

With architecture, you can inspect the change.

Regression Is Part of Improvement

Not every change is an improvement.

A new prompt can be worse. A larger context can add noise. A memory can become stale. A critic can push the system toward blandness. A new model can handle one domain better and another worse. A decentralized team can create more confusion than a single agent.

Agent systems therefore need regression awareness:

current version
  ↓
candidate change
  ↓
comparison
  ↓
accept, reject, or roll back

This is an engineering concern and part of the working relationship. If an agentic system is going to support a person over time, it must preserve what works. Growth without memory of quality becomes drift.

    flowchart TD
    A[Current version]
    B[Proposed change]
    C[Evaluate candidate]
    D[(Historical performance data)]
    E{Better than current version?}
    F[Accept updated version]
    G[Rollback or keep baseline]

    A --> B
    B --> C
    D --> C
    C --> E
    E -->|Yes| F
    E -->|No| G
    G --> A
  

The Pattern So Far

A well-designed agentic system is built from mechanisms that can be named:

1. Roles shape instructions, context, tools, authority, and criteria.
2. Tools extend what the runtime can observe or do.
3. Memory stores and retrieves selected information without changing model weights.
4. Steps turn hidden process into inspectable intermediate state.
5. Reflection generates candidate improvements that still need judgment.
6. Coordination connects roles through structured handoffs.
7. Emergence arises from interaction between components.
8. Adaptation comes from changes in context, state, memory, tools, configuration, or training.
9. Regression awareness protects useful behavior from being accidentally lost.

The point is not to make agents sound more human. It is to make agent behavior easier to shape.

These mechanisms are easier to understand once you build them, even in miniature. A manager, a researcher, a few tool-like prompts, and a reflection loop are enough to make the architecture visible from an ordinary chat.