An AI agent often fails for a surprisingly ordinary reason:

it commits too early.

It finds one plausible next action, follows it, and then spends the rest of the run trying to make that first choice work.

That can look intelligent because the agent keeps reasoning, calling tools, revising plans, and explaining itself.

But underneath, the trajectory may be almost completely determined by an early mistake.

A coding agent chooses the wrong implementation strategy and spends twenty tool calls repairing it.

A research agent finds the first plausible explanation and gathers only evidence that supports it.

A browser agent chooses the wrong navigation path and keeps trying to recover from increasingly confusing page state.

A data agent commits to the wrong transformation and then compensates with more transformations.

The problem is not necessarily that the model cannot produce a better solution.

The better solution may simply never get explored.

The next step in our agent stack is therefore search.

Not web search.

Search over possible agent trajectories.

Instead of:

problem
  ↓
first plausible action
  ↓
second plausible action
  ↓
final answer

we allow multiple possibilities to remain alive:

                  state 0
              /      |      \
             A       B       C
            / \     / \     / \
          A1  A2   B1  B2   C1  C2

Then we evaluate, prune, and continue only with promising branches.

This sounds like a large conceptual jump.

It is actually a small extension of several techniques we have already built.

We already know how to:

• generate multiple candidates,
• score candidates,
• critique candidates,
• track state,
• verify outcomes,
• bound loops,
• record trajectories.

Search simply changes where we branch and which branches we continue.

That makes it one of the most useful transitions from a reactive agent into a deliberate agent.

The Search Problem Behind Many Agent Failures

Suppose we ask a coding agent to fix a failing test.

A simple agent might do this:

inspect error
   ↓
choose likely file
   ↓
edit file
   ↓
run tests
   ↓
repair consequences

That is one trajectory.

But there may have been several plausible hypotheses at the first step:

failure
  ├── implementation bug
  ├── bad test fixture
  ├── stale generated file
  └── configuration mismatch

If the agent chooses implementation bug immediately, the rest of the run may be spent inside the wrong branch.

A search-based agent can preserve alternatives:

failure
  ↓
generate hypotheses
  ↓
score / test cheaply
  ↓
expand top hypotheses
  ↓
execute stronger checks
  ↓
commit later

The key idea is simple:

Delay commitment when several plausible trajectories exist and cheap evidence can distinguish them.

Search is useful when uncertainty is about what path to follow, not merely how to phrase an answer.

Best-of-N Was Search, but Only at the End

Earlier in this series we built Best-of-N:

prompt
  ↓
answer A
answer B
answer C
  ↓
score
  ↓
select best

That is already a form of search.

But the candidates are usually complete outputs.

Suppose each complete solution requires ten expensive tool calls.

Generating five complete trajectories costs roughly five times as much work.

A more efficient approach is to branch earlier:

initial state
  ↓
three possible approaches
  ↓
cheap evaluation
  ↓
keep two
  ↓
expand each
  ↓
keep one

This is incremental search.

We spend compute where evidence suggests it is useful.

The Smallest Search Agent

We can start with a deliberately tiny abstraction.

from dataclasses import dataclass, field


@dataclass
class SearchNode:
    state: dict
    actions: list[str] = field(default_factory=list)
    score: float = 0.0
    depth: int = 0

Each node represents one possible point in the agent trajectory.

The search loop needs only four operations:

select
expand
evaluate
prune

A simple beam-style search looks like this:

def search(initial_state, expand, evaluate, beam_width=3, max_depth=5):
    frontier = [SearchNode(state=initial_state)]

    for depth in range(max_depth):
        children = []

        for node in frontier:
            for action, next_state in expand(node.state):
                child = SearchNode(
                    state=next_state,
                    actions=node.actions + [action],
                    depth=node.depth + 1,
                )
                child.score = evaluate(child)
                children.append(child)

        if not children:
            break

        children.sort(key=lambda n: n.score, reverse=True)
        frontier = children[:beam_width]

    return max(frontier, key=lambda n: n.score)

Nothing here requires an LLM.

The agent-specific pieces are hidden behind:

expand(state)

and:

evaluate(node)

That separation matters.

Search is a runtime mechanism.

The LLM may propose expansions or score nodes, but it is not the search algorithm itself.

Search Is About Partial Solutions

The major conceptual difference from Best-of-N is this:

We evaluate unfinished trajectories.

For a coding agent, a partial node might be:

Hypothesis: cache invalidation bug
Evidence: failure disappears when cache disabled
Next action: inspect cache key construction

For a research agent:

Hypothesis: productivity decline is driven by demographic composition
Evidence collected: two supporting sources
Missing evidence: sector-level decomposition

For a browser agent:

Current page: checkout
Progress: address entered
Alternative next actions:
- click continue
- fix postcode validation

The search algorithm decides which unfinished states deserve more compute.

Search Requires a State Representation

Search becomes difficult if the agent state is only a giant conversation transcript.

A node should contain the information necessary to continue the trajectory.

For example:

state = {
    "goal": "Fix failing parser test",
    "hypothesis": "delimiter handling regression",
    "files_read": ["parser.py", "tests/test_parser.py"],
    "tests": {
        "test_pipe_delimiter": "failed",
    },
    "changes": [],
    "remaining_budget": 8,
}

Now two branches can diverge cleanly.

Branch A:

hypothesis = parser logic regression

Branch B:

hypothesis = test fixture regression

Without explicit state, branches easily contaminate one another.

This is another reason structured runtime state matters more as agent sophistication increases.

Expansion: How Do We Create Alternatives?

The expand() function generates possible next moves.

At the simplest level:

def expand(state):
    return model.propose_actions(state, n=3)

But we can make expansion more deliberate.

For example:

one conservative action
one diagnostic action
one alternative hypothesis

That creates diversity by construction.

A coding agent might expand with:

1. inspect the most likely implementation
2. run a discriminating test
3. inspect configuration or environment

A research agent might generate:

1. strengthen current hypothesis
2. seek contradictory evidence
3. investigate a competing explanation

This is much better than asking:

Give me three more ideas.

The search policy should generate meaningfully different branches.

Duplicate Branches Waste Search Budget

A common search failure is fake diversity.

Suppose the model proposes:

A: inspect parser.py
B: inspect the parser implementation
C: open parser.py and examine delimiter logic

Those are not three branches.

They are one branch written three ways.

We need branch deduplication.

A cheap version can normalize action signatures:

def action_fingerprint(action: dict) -> tuple:
    return (
        action["tool"],
        tuple(sorted(action.get("args", {}).items())),
    )

Semantic branches may require stronger deduplication:

embedding similarity
state similarity
shared changed files
shared hypothesis label

A useful metric is:

unique_branch_ratio = unique_branches / generated_branches

If you generate 20 candidates but only 4 are meaningfully distinct, your search width is not really 20.

Evaluation: The Hardest Part of Search

Generating alternatives is usually easier than evaluating unfinished ones.

The evaluation function is the search heuristic.

Bad heuristic:

score = llm("How promising is this approach?")

Better evaluation combines evidence.

For a coding agent:

+ tests fixed
+ failing tests reduced
+ static checks pass
+ patch size reasonable
- new failures introduced
- unsupported assumptions

For a research agent:

+ independent sources
+ claim coverage
+ source quality
+ contradictory evidence addressed
- unsupported claims
- duplicated sources

For browser automation:

+ task fields completed
+ navigation closer to target
- validation errors
- repeated page states

The evaluator does not need to predict the final result perfectly.

It only needs to rank partial trajectories well enough that promising branches receive more compute.

Objective Signals Should Dominate When Available

If a branch can be tested, test it.

For code:

run targeted tests

For SQL:

execute query against fixtures

For browser tasks:

inspect DOM state

For data transformations:

run schema validation

For planning:

check constraints

A model’s opinion about whether a branch is promising should generally be weaker evidence than direct environment feedback.

This gives us a search policy like:

objective verifier
      ↓
heuristic metrics
      ↓
learned scorer
      ↓
LLM judge

The exact order depends on the application, but the principle is consistent:

Use the cheapest reliable evidence first.

Beam Search: Keep the Best Few Paths Alive

One of the simplest useful search strategies is beam search.

Instead of keeping every branch, retain the top k after each expansion.

beam width = 2

           root
        /   |   \
       A    B    C
       ↓    ↓
      A1    B1
     / \    / \
   A2 A3  B2 B3
      ↓    ↓
     top two

This controls combinatorial explosion.

If each node generates four children and we search five levels deep, exhaustive search explores:

4⁵ = 1024 terminal trajectories

A beam width of three keeps only three active nodes per depth.

That turns exponential growth into something much closer to:

beam_width × branching_factor × depth

This is not free, but it is manageable.

Search Width and Search Depth Are Different Budgets

There are two basic ways to spend more inference.

Breadth:

try more alternatives

Depth:

follow each alternative further

Different tasks want different allocations.

For a coding diagnosis problem, breadth may matter initially:

several plausible root causes

Once one root cause has strong evidence, depth becomes more useful:

implement + test + refine

For research, we may deliberately maintain breadth longer because competing hypotheses are valuable.

Search policy therefore includes a compute-allocation decision.

Search Explosion: The Most Obvious Failure

Naive search can become catastrophically expensive.

Suppose:

branching factor = 5
search depth = 8

Full expansion creates:

5⁸ = 390,625

terminal trajectories.

That is not an agent architecture.

That is a token furnace.

Real search agents need explicit controls:

max nodes
max depth
beam width
max model calls
max tool calls
wall-clock budget
cost budget
no-improvement patience

For example:

class SearchBudget:
    def __init__(self, max_nodes=50, max_depth=6, max_calls=40):
        self.max_nodes = max_nodes
        self.max_depth = max_depth
        self.max_calls = max_calls

Search should always know how much search it is allowed to perform.

When Should Search Stop?

A search agent should not continue merely because branches remain.

Useful stopping conditions include:

verified solution found
score threshold reached
no improvement for N expansions
all branches invalid
budget exhausted
branches converged
expected gain below cost

The last one is especially important.

Suppose:

beam width 1 → 74% success
beam width 2 → 81%
beam width 4 → 82%
beam width 8 → 82.3%

Search width eight may be unjustified.

The relevant metric is not merely quality.

It is quality relative to additional inference cost.

Search Should Earn Its Compute

A useful experiment sweeps search budget.

strategy     success    model calls    latency
------------------------------------------------
one-shot       70%          1            2s
Best-of-3      78%          3            4s
beam-2         83%          8           10s
beam-4         84%         16           18s
beam-8         84%         31           35s

Now we can see the knee in the curve.

The question is not:

Does search help?

The question is:

Where does additional search stop paying for itself?

A Better Search Metric: Cost Per Verified Success

Suppose two systems have:

Agent A
success = 80%
cost/run = $0.10

Agent B
success = 90%
cost/run = $0.40

A useful comparison is:

cost per successful task

Approximately:

A = 0.10 / 0.80 = $0.125
B = 0.40 / 0.90 = $0.444

Agent B is more capable but much less efficient.

Depending on the application, either may be correct.

For an expensive production incident, the extra compute may be trivial.

For millions of low-value support requests, it may be unacceptable.

Search Is Not the Same as Retry

Retry says:

that failed
try again

Search says:

before committing, preserve multiple plausible trajectories

Retry is reactive.

Search is exploratory.

They can be combined, but they solve different problems.

Search Is Not the Same as Critique and Revision

Critique/revision usually follows one trajectory:

draft
 ↓
critique
 ↓
revision
 ↓
revision

Search keeps alternatives alive:

          draft
       /    |    \
      A     B     C
      ↓     ↓
     A1    B1

Critique/revision is primarily depth.

Search adds breadth.

A strong agent often uses both:

generate branches
      ↓
select promising branches
      ↓
revise each
      ↓
verify

Search Is Not Yet MCTS

You will often see search agents described immediately in terms of Monte Carlo Tree Search.

That is an advanced technique.

We do not need it yet.

The core idea is already visible with beam search:

generate alternatives
evaluate partial states
allocate compute selectively
prune weak branches

MCTS adds more sophisticated selection and value propagation.

We will reserve that for Advanced Agents From First Principles.

The simpler search mechanism is easier to debug and often sufficient.

Application: Coding Agents

Coding agents are one of the clearest uses of search.

A bug rarely has exactly one plausible explanation.

A useful search state may include:

hypothesis
files inspected
tests executed
patches attempted
failures introduced
current verifier score

A branch could represent:

Hypothesis A: parser implementation regression
Hypothesis B: fixture mismatch
Hypothesis C: dependency-version change

Cheap branch evaluation:

run targeted test
inspect blame/diff
check environment version

Expensive evaluation:

implement patch
run full suite

The search agent should use cheap evidence before paying for expensive modifications.

Common coding-agent failure

agent edits code immediately

Better:

agent first generates competing diagnoses
      ↓
runs discriminating checks
      ↓
commits to strongest diagnosis
      ↓
edits code

This reduces destructive trial-and-error.

Application: Research Agents

Research agents benefit from search for a different reason.

The danger is not only wrong execution.

It is premature hypothesis commitment.

Instead of:

question
 ↓
first plausible explanation
 ↓
find supporting sources

use:

question
 ↓
competing hypotheses
 ↓
source search per hypothesis
 ↓
contradictory evidence
 ↓
rank explanations

A branch can represent a hypothesis plus its evidence set.

Search scores might include:

source quality
independence
claim coverage
contradiction handling
unresolved questions

This is particularly useful in technical investigations, economic analysis, literature reviews, and debugging research claims.

Application: Browser Agents

Browser agents operate in a partially observable environment.

A button label may be ambiguous.

Several navigation strategies may exist.

Search can preserve alternative interpretations:

checkout page
  ├── continue button advances
  ├── validation error blocks progress
  └── address selector must be completed first

Instead of repeatedly clicking the same thing, the agent can evaluate branches using DOM state.

Useful node score:

required fields completed
validation errors reduced
page progress increased
repeated states penalized

Search is particularly useful when navigation is reversible or cheap to simulate.

It is much less attractive when actions have irreversible side effects.

Application: Data and Analytics Agents

Suppose an agent must repair a broken data pipeline.

Several transformations may plausibly fix a schema mismatch:

cast column
rename field
change join key
fill missing values

Search can evaluate them against deterministic validators.

candidate transformation
       ↓
run on fixture
       ↓
schema validation
       ↓
row-count invariants
       ↓
quality score

This is a particularly strong application because the environment provides objective feedback.

The agent does not need to guess whether the branch worked.

Application: Planning and Scheduling Agents

Search is useful when there are several valid plans under constraints.

For example:

schedule tasks
allocate workers
respect dependencies
minimize lateness

A branch represents a partial plan.

The evaluator can reject constraint violations immediately.

hard constraints → prune
soft objectives → score

This is often much better than asking an LLM to produce one final schedule and then hoping it satisfies everything.

Application: DevOps and Incident Agents

Incident response often involves competing hypotheses:

CPU saturation
bad deployment
network issue
database contention
cache failure

A search agent can prioritize diagnostic actions that distinguish hypotheses.

inspect metrics
check deployment diff
query DB locks
inspect upstream latency

The best search action may not fix the system immediately.

It may simply produce the most information.

That introduces an important idea:

Some actions are valuable because they reduce uncertainty.

This will become even more important in advanced agent search.

Search Actions Can Be Informational or Transformational

We can divide actions into two broad classes.

Informational

read file
run diagnostic
search docs
inspect metrics
query database

Transformational

edit file
deploy
submit form
change database
send message

Search generally works best when informational actions are cheap and transformational actions are expensive or risky.

A strong policy may prefer information-gathering branches early.

uncertain state
    ↓
cheap diagnostic actions
    ↓
confidence rises
    ↓
commit to transformation

This is a practical way to make agents less impulsive.

Branching on Irreversible Actions Is Dangerous

Search assumes branches can be evaluated independently.

That is easy in simulation.

It is harder in the real world.

You cannot:

send three alternative refunds

and later keep the best one.

For irreversible tools, search should happen before execution.

For example:

generate three refund plans
 ↓
validate policy
 ↓
select one
 ↓
execute once

This is a crucial boundary.

Search over plans or simulated effects when real actions cannot be undone safely.

Search Needs Provenance

Every branch should know where it came from.

@dataclass
class SearchNode:
    id: str
    parent_id: str | None
    depth: int
    action: dict | None
    observation: dict | None
    score: float
    state: dict

This gives us a trajectory tree.

Now debugging can answer:

Which branch produced this result?
Why was another branch pruned?
Which evaluator score caused the decision?
Where did branches diverge?

Without lineage, search becomes very difficult to inspect.

Log the Search Frontier

A normal agent trace logs one trajectory.

A search agent should log the frontier.

For every expansion:

node ID
parent ID
depth
action
score
verification results
prune reason
cost

Useful summary metrics include:

nodes generated
nodes expanded
nodes pruned
unique branches
max depth
average branching factor
score improvement by depth
verified successes
search cost

Now search becomes observable software rather than invisible model reasoning.

Search Failure Mode: Weak Heuristic

Suppose the correct branch appears early but gets a mediocre partial score.

The search prunes it.

The final result fails.

This is search error caused by the evaluator.

We can measure something similar to the oracle@N idea from Best-of-N.

If a successful trajectory existed somewhere in generated branches but was pruned, the generator was not the main problem.

The search policy was.

Useful diagnostic:

oracle_generated_success
vs
selected_success

The gap measures search-selection failure.

Search Failure Mode: Search Becomes Model Self-Confirmation

If the same model:

generates branches
scores branches
explains why the winner is best

then diversity may be superficial.

The same bias can dominate every stage.

Whenever possible, introduce independent evidence:

tests
runtime output
retrieved facts
constraints
learned scorer
separate evaluator

Search quality depends on the independence of its feedback.

Search Failure Mode: Exploration Without Exploitation

An agent can also search too much.

It keeps generating new ideas instead of finishing one.

A
B
C
D
E
F
...

This is the opposite of premature commitment.

We need a balance:

explore alternatives
        ↓
identify promising region
        ↓
exploit / deepen

Beam search handles this crudely by pruning.

More advanced algorithms handle it explicitly.

For now, a simple policy is enough:

wide early
narrow later

Adaptive Beam Width

We do not need the same search width at every depth.

For example:

depth 0 → keep 5 hypotheses
depth 1 → keep 3
depth 2 → keep 2
depth 3 → keep 1

This mirrors how humans often solve problems:

consider several explanations
collect evidence
narrow possibilities
commit

It also reduces cost.

Confidence Is Not Enough

A model saying:

I am 95% confident

should not automatically collapse the search frontier.

Confidence can be poorly calibrated.

Better collapse conditions are evidence-based:

one hypothesis explains all observations
competing branches fail deterministic checks
verification threshold reached
score margin is stable across evaluators

Search should close because the evidence discriminated between branches, not merely because the model sounded certain.

A Practical Search-Agent Skeleton

Here is a compact version tying the pieces together.

from dataclasses import dataclass, field
from typing import Any, Callable
import uuid


@dataclass
class Node:
    state: dict[str, Any]
    score: float = 0.0
    depth: int = 0
    parent_id: str | None = None
    id: str = field(default_factory=lambda: str(uuid.uuid4()))
    action: dict[str, Any] | None = None
    observation: dict[str, Any] | None = None


def beam_search(
    initial_state: dict,
    propose: Callable,
    execute: Callable,
    evaluate: Callable,
    is_success: Callable,
    beam_width: int = 3,
    branch_factor: int = 3,
    max_depth: int = 5,
):
    frontier = [Node(state=initial_state)]
    all_nodes = list(frontier)

    for _ in range(max_depth):
        children = []

        for node in frontier:
            actions = propose(node.state, branch_factor)

            for action in actions:
                observation, next_state = execute(node.state, action)

                child = Node(
                    state=next_state,
                    depth=node.depth + 1,
                    parent_id=node.id,
                    action=action,
                    observation=observation,
                )

                child.score = evaluate(child)
                children.append(child)
                all_nodes.append(child)

                if is_success(child):
                    return child, all_nodes

        if not children:
            break

        children.sort(key=lambda n: n.score, reverse=True)
        frontier = children[:beam_width]

    return max(frontier, key=lambda n: n.score), all_nodes

This deliberately omits many production concerns.

But the mechanism is visible:

branch
execute or simulate
evaluate
prune
repeat

Search Before Side Effects

The skeleton above calls execute() on every branch.

That is only safe if actions are reversible, simulated, read-only, isolated, or otherwise branch-safe.

In production, often use:

propose plan
 ↓
simulate / validate
 ↓
score
 ↓
select
 ↓
execute chosen action once

For coding agents, separate worktrees or sandboxes can make branches executable.

For database agents, use transactions or fixtures.

For browser agents, use independent sessions where appropriate.

For irreversible business actions, do not execute competing branches in production.

Real Software Application Matrix

The same search mechanism appears in each system.

What changes is:

state
branch representation
evaluator
side-effect boundary
success condition

If Your Agent Picks the First Plausible Solution

A practical diagnostic sequence:

1. Check whether alternatives actually exist

If the task has one obvious deterministic path, search is unnecessary.

2. Generate alternatives before acting

Ask for distinct strategies, not paraphrases.

3. Measure branch diversity

If branches collapse to the same action, fix expansion first.

4. Add cheap discriminating tests

The search heuristic needs evidence.

5. Delay irreversible actions

Search plans before executing side effects.

6. Bound width and depth

Never allow unbounded expansion.

7. Log why branches were pruned

If the right answer was generated and discarded, the evaluator is the problem.

8. Compare against simpler baselines

Always benchmark against:

one-shot
Best-of-N
critique/revision

Search should earn its complexity.

Experiment: Does Search Actually Help?

Build a dataset of tasks with known success criteria.

Compare:

A: one-shot trajectory
B: Best-of-3 final answers
C: critique/revision
D: beam search width 2
E: beam search width 4

Hold constant where possible:

model
tool set
prompt information
verification rules
task set

Measure:

verified success
oracle generated success
pruning regret
model calls
tool calls
nodes expanded
latency
cost per successful task

Also inject tasks where the first plausible strategy is deliberately wrong.

That tests whether the system genuinely benefits from preserving alternatives.

Search Changes the Meaning of Agent Intelligence

Without search, the model often determines the trajectory in a single sequence.

model choice
   ↓
next state
   ↓
model choice

With search, the runtime becomes more important.

model proposes alternatives
        ↓
runtime preserves branches
        ↓
environment provides evidence
        ↓
runtime allocates compute
        ↓
model proposes next expansions

The intelligence of the system is no longer located entirely in the next-token prediction.

It is partly in the control structure around the model.

That is one of the central ideas of agent engineering.

Do You Actually Need Search?

Use search when:

multiple plausible paths exist
+
early choices strongly affect later outcomes
+
partial trajectories can be evaluated
+
additional inference is affordable

Do not add search merely because it sounds sophisticated.

If:

the task has a deterministic workflow

use normal code.

If:

the final answer is cheap to sample

Best-of-N may be enough.

If:

the first draft is usually close

critique/revision may be enough.

If:

multiple distinct paths must remain alive

search begins to justify itself.

Where We Are Now

Our agent has grown considerably.

structured actions
      ↓
Best-of-N
      ↓
critique + revision
      ↓
planning
      ↓
loop control
      ↓
tool routing
      ↓
memory
      ↓
search

But there is still a dangerous weakness.

The agent may produce an impressive trajectory, call the right tools, search several branches, and confidently tell us:

Done.

while the task has actually failed.

The next step is therefore one of the most important in the entire series:

Do not ask the agent whether it succeeded. Verify the result outside the model.

Next:

Agents From First Principles 09: AI Agent Says It Worked When It Didn’t? Verify the Result Outside the LLM.