An agent can have a perfectly capable model and still behave badly because its tools are badly designed.

This is one of the most common agent failures in production:

user goal
   ↓
agent
   ↓
wrong tool
   ↓
wrong action

The model may understand the task.

The agent may have enough context.

The problem is that the action space is ambiguous.

If two tools overlap, their descriptions are vague, their schemas are huge, or their results are difficult to interpret, the model has to guess.

And once the model guesses wrong, the rest of the loop can be perfectly implemented and still fail.

This post is about fixing that boundary.

The central idea is simple:

A tool is not just a function the model can call. It is part of the agent’s decision space.

That means tool design is agent design.

The failure people actually see

Suppose an agent can call these two tools:

def search(query: str):
    ...


def lookup(query: str):
    ...

What is the difference?

A human reading the code may know.

The model does not.

If the descriptions are also vague:

search: search for things
lookup: look things up

then choosing the right tool is mostly a language-model guess.

The first fix is therefore not a better model.

It is a better action space.

The tool-selection pipeline

A useful tool-using agent can be decomposed into:

goal
 ↓
available tools
 ↓
tool descriptions + schemas
 ↓
router / policy
 ↓
selected tool
 ↓
argument validation
 ↓
execute
 ↓
structured result
 ↓
state update

Each stage can fail independently.

That matters because “the agent chose the wrong tool” is too vague to debug.

We want to ask:

Was the right tool available?
Was its description discriminative?
Was the wrong tool too similar?
Were the arguments valid?
Did execution succeed?
Did the agent interpret the result correctly?

That gives us something measurable.

1. Start with a finite action space

Do not begin with a generic tool like:

def do_anything(action: str, payload: dict):
    ...

That gives the model an enormous semantic space to invent inside.

Prefer narrow tools:

def read_file(path: str) -> dict:
    ...


def edit_file(path: str, old: str, new: str) -> dict:
    ...


def run_tests(target: str) -> dict:
    ...

Now the model chooses among a small number of explicit capabilities.

This is the same principle we used earlier with structured actions:

probabilistic model
        ↓
small deterministic interface

The runtime should make invalid actions difficult to express.

2. Tool descriptions are part of the routing policy

Consider these descriptions:

read_file
Read a file.

search_files
Search files.

They are technically correct.

They are operationally weak.

A better version is:

read_file
Return the complete contents of one known file path.
Use this when the exact file path is already known.
Do not use it to discover files.

search_files
Search repository files for a filename, symbol, phrase or pattern.
Use this when the relevant file path is unknown.
Do not use it when the exact path is already known.

Notice what changed.

We did not merely describe what each function does.

We described when to choose it and when not to choose it.

That makes the routing boundary much clearer.

A useful tool description therefore contains:

what the tool does
when to use it
when not to use it
what inputs it expects
what output it returns
important side effects

3. Overlapping tools create routing ambiguity

Suppose a coding agent exposes:

search_code
find_code
lookup_symbol
repository_search
find_file

Five tools may look richer than one.

But if their semantic boundaries overlap, the model has a harder classification problem.

A useful way to think about tool selection is:

task description
      ↓
classification over available actions

If the classes overlap heavily, classification gets harder.

This suggests a practical rule:

Prefer fewer tools with sharp semantic boundaries over many tools with fuzzy boundaries.

This does not mean one giant tool.

It means each tool should represent a distinct action.

4. Tool count itself can become a problem

Imagine giving the model 80 tools.

Even if every tool is valid, several new costs appear:

more prompt/schema tokens
more candidate actions
more overlapping descriptions
more routing mistakes
more difficult debugging

The naive architecture is:

agent
 ↓
all 80 tools

A better architecture can be hierarchical:

agent
 ↓
choose domain
 ├─ repository
 ├─ browser
 ├─ database
 └─ deployment
       ↓
   choose tool

For example:

coding task
   ↓
repository tool group
   ↓
read_file / search_code / edit_file / run_tests

The first routing step reduces the second routing problem.

This does not automatically require another LLM call.

The first routing layer can be deterministic when context already tells us which tool family is relevant.

5. Tool schemas should constrain, not merely document

Consider:

{
    "tool": "run_tests",
    "args": {
        "whatever": "pytest maybe"
    }
}

That is structured JSON.

It is still a weak interface.

A stronger schema might be:

RUN_TESTS_SCHEMA = {
    "target": str,
    "mode": {"unit", "integration", "all"},
}

Then the runtime can reject:

{
    "target": 42,
    "mode": "fast-ish"
}

before anything executes.

The model proposes.

The runtime validates.

The executor acts only after validation succeeds.

6. Separate syntactic, structural and semantic validity

An action can be valid at several different levels.

Syntax

Is the model output parseable?

{"tool": "read_file", "args": {"path": "src/app.py"}}

Structure

Does it contain the required fields with correct types?

tool: string
args: object

Semantics

Does the action make sense?

Does the path exist?
Is the path inside the repository?
Is this operation allowed?

Policy

Is this action authorized in the current context?

read_file       allowed
edit_file       allowed
production_deploy denied

These checks belong in runtime code where possible.

Do not ask the model:

Is this tool call valid?

when the program can answer deterministically.

7. A small tool registry

We can make the action space explicit:

from dataclasses import dataclass
from typing import Callable


@dataclass
class Tool:
    name: str
    description: str
    handler: Callable
    required_args: dict[str, type]


TOOLS = {
    "read_file": Tool(
        name="read_file",
        description=(
            "Read one known file path. Use when the exact path is known. "
            "Do not use for file discovery."
        ),
        handler=read_file,
        required_args={"path": str},
    ),
    "search_code": Tool(
        name="search_code",
        description=(
            "Search repository content when the exact file is unknown. "
            "Use for symbols, phrases and patterns."
        ),
        handler=search_code,
        required_args={"query": str},
    ),
}

Now validation can be generic.

def validate_tool_call(tool_name: str, args: dict):
    if tool_name not in TOOLS:
        raise ValueError(f"unknown tool: {tool_name}")

    tool = TOOLS[tool_name]

    for name, expected_type in tool.required_args.items():
        if name not in args:
            raise ValueError(f"missing argument: {name}")

        if not isinstance(args[name], expected_type):
            raise TypeError(
                f"{name} must be {expected_type.__name__}"
            )

    unknown = set(args) - set(tool.required_args)
    if unknown:
        raise ValueError(f"unexpected arguments: {sorted(unknown)}")

This is deliberately boring.

Boring boundaries are useful in agent systems.

8. Do not let the model invent tool names

A common failure looks like:

{
  "tool": "read_repository_file_safely",
  "args": {"path": "main.py"}
}

The model has invented a plausible tool.

Do not fuzzy-match that automatically to read_file unless you have explicitly designed and tested that behaviour.

The safe default is:

unknown tool
 ↓
reject
 ↓
return available tool names
 ↓
request a corrected action

For example:

def tool_error(tool_name: str):
    return {
        "ok": False,
        "error": "unknown_tool",
        "requested": tool_name,
        "available": sorted(TOOLS),
    }

The correction loop receives evidence rather than a vague message.

9. Tool results should be structured too

Tool inputs are often structured.

Tool outputs should be as well.

Weak result:

Looks like the tests mostly passed but two things went wrong.

Better:

{
    "ok": False,
    "passed": 184,
    "failed": 2,
    "errors": [
        "tests/test_auth.py::test_expired_token",
        "tests/test_api.py::test_unauthorized",
    ],
}

Why?

Because the result becomes the next observation.

If the next decision depends on fuzzy prose, we inject unnecessary ambiguity back into the loop.

A useful pattern is:

structured action
      ↓
deterministic execution
      ↓
structured observation

The model can still receive a human-readable rendering, but the runtime should retain the structured representation.

10. Separate tool selection from tool execution

Do not implement:

response = model(prompt)
execute(response)

Use:

response = model(prompt)

action = parse(response)
validate(action)
authorize(action)
result = execute(action)

This separation gives us hooks for:

logging
validation
permissions
rate limits
confirmation
retries
simulation
dry-run
policy enforcement

The model does not own the side effect.

The runtime does.

11. Tool routing can be deterministic

Not every tool choice needs an LLM.

Suppose the task state says:

state.current_phase == "verification"

and only one verification tool is legal:

run_tests

Then calling the model to choose between all tools adds unnecessary uncertainty.

We can simply route:

if state.current_phase == "verification":
    allowed_tools = ["run_tests"]

The model chooses only when there is a genuine decision to make.

This is important.

Agent systems are not improved by maximizing the number of decisions delegated to the model.

They are often improved by minimizing unnecessary decisions.

12. Restrict tools dynamically

A planner may produce:

1. locate relevant code
2. inspect implementation
3. edit implementation
4. run tests

There is no reason to expose deployment tools during step 1.

We can narrow the action space by state:

def allowed_tools(state):
    if state.phase == "discover":
        return ["search_code", "read_file"]

    if state.phase == "modify":
        return ["read_file", "edit_file"]

    if state.phase == "verify":
        return ["run_tests", "read_file"]

    return []

This reduces routing ambiguity and limits accidental side effects.

13. When should we add a dedicated router?

At small scale:

model sees tools
 ↓
model chooses tool

is often enough.

As the tool space grows, we can split routing:

user task
 ↓
router
 ↓
relevant tool subset
 ↓
agent policy
 ↓
exact tool

The router itself can be:

deterministic rules
classifier
embedding similarity
small local model
large language model
learned policy

The important thing is not the sophistication of the router.

It is whether routing accuracy improves enough to justify the additional machinery.

14. Measure routing instead of guessing

Create a small labelled dataset:

cases = [
    {
        "task": "Find where UserService is defined",
        "expected_tool": "search_code",
    },
    {
        "task": "Open src/users/service.py",
        "expected_tool": "read_file",
    },
    {
        "task": "Run the authentication unit tests",
        "expected_tool": "run_tests",
    },
]

Then measure:

tool-selection accuracy
unknown-tool rate
argument-validity rate
execution-success rate
recovery rate
mean tool calls per task
latency
cost

Now tool design can be benchmarked.

For example:

baseline descriptions
        vs
explicit use / do-not-use descriptions
        vs
state-restricted tool set
        vs
dedicated router

This tells us whether the routing mechanism actually works.

15. Confusion matrices are useful for agents too

Suppose the expected tools are:

read_file
search_code
run_tests

We might discover:

                predicted
             read search tests
expected read   84   15    1
expected search 23   75    2
expected tests   1    2   97

Now the problem is obvious.

read_file and search_code overlap semantically.

Instead of changing the entire agent, we can improve those two interfaces.

This is much better than saying:

The model is bad at tools.

16. A tool can succeed while the task fails

Consider:

agent chooses search_code
 ↓
search_code succeeds
 ↓
returns irrelevant files

The tool executed successfully.

The action was still unhelpful.

So we need to distinguish:

routing success
argument validity
execution success
observation usefulness
goal progress

These are different metrics.

A production agent should not collapse all of them into one success=True flag.

17. Tool success should feed the loop controller

The previous post introduced progress and stopping conditions.

Tool results now become part of that mechanism.

Example:

result = execute(action)

state.tool_calls += 1
state.history.append((action, result))

if result["ok"]:
    state.successful_tool_calls += 1
else:
    state.failed_tool_calls += 1

But execution success alone is not progress.

For a coding agent:

read_file succeeded

may not mean anything improved.

Better progress signals might be:

relevant symbol located
patch created
failing tests decreased
requested behaviour verified

This connects tool design to the broader agent loop.

Where these ideas appear in real software

The pattern is generic, but the tool interfaces are domain-specific.

Coding agents

Typical tools:

search_code
read_file
edit_file
run_tests
inspect_diff
git_status

Common routing failures:

reading random files instead of searching first
editing before understanding dependencies
running the full test suite when a targeted test exists
repeatedly searching after the symbol is already known

Good design makes the distinction explicit:

search_code → discover location
read_file   → inspect known location
edit_file   → mutate known file
run_tests   → verify behaviour

This is exactly the kind of environment where narrow tools and state-dependent exposure work well.

Browser agents

Typical tools:

navigate
click
type
select
extract
scroll

Common failure:

agent uses click when it should type
agent repeatedly clicks a disabled button
agent searches the page when navigation is required

Useful structured observations include:

{
    "url": "...",
    "element_found": True,
    "element_enabled": False,
    "page_changed": False,
}

That is much more useful than:

The click probably did not work.

Research agents

Typical tools:

search_web
open_source
find_in_source
extract_claim
record_citation

The important distinction is often:

discovery
   vs
source inspection
   vs
evidence extraction

A weak research agent repeatedly searches when it should already be reading the sources it found.

Tool boundaries can encode that progression.

Customer-support agents

Typical tools:

lookup_customer
lookup_order
check_refund_policy
issue_refund
escalate_case

Here the action boundary also becomes an authority boundary.

Reading an order and issuing a refund should not have equivalent permissions.

A useful architecture is:

read-only tools
      ↓
policy checks
      ↓
side-effect tools
      ↓
confirmation / authorization

The tool registry can carry those capabilities explicitly.

Data and analytics agents

Typical tools:

inspect_schema
run_query
profile_data
transform_data
validate_output
write_dataset

Common failure:

agent writes before validating
agent reruns expensive queries unnecessarily
agent chooses transformation tools before inspecting schema

State-restricted tool sets are especially useful here.

DevOps agents

Typical tools:

inspect_logs
check_metrics
restart_service
deploy
rollback

These tools have radically different risk levels.

The agent should not see them as flat alternatives.

A safer action hierarchy is:

observe
  ↓
diagnose
  ↓
propose remediation
  ↓
authorize
  ↓
execute
  ↓
verify

restart_service and rollback may require stronger gates than inspect_logs.

The model can propose them.

The runtime still owns permission.

Application matrix

The same underlying agent mechanism appears in all of them.

The useful tool boundaries differ.

A complete minimal tool-routing loop

Here is the basic shape:

def run_agent(goal, model, state):
    while not state.done:
        tools = allowed_tools(state)

        prompt = build_prompt(
            goal=goal,
            state=state,
            tools=tools,
        )

        raw = model(prompt)
        action = parse_action(raw)

        validate_tool_call(action.tool, action.args)
        authorize(action, state)

        result = TOOLS[action.tool].handler(**action.args)

        observation = normalize_result(action.tool, result)
        state.observe(action, observation)

        if verify_goal(goal, state):
            state.done = True
            state.stop_reason = "goal_verified"

        elif state.no_progress():
            state.done = True
            state.stop_reason = "no_progress"

    return state

Notice what the model does not own:

which tools exist
whether arguments are valid
whether an action is authorized
how the tool executes
whether the goal is actually complete

That separation is one of the foundations of reliable agent software.

Debugging: the agent keeps choosing the wrong tool

Start with evidence.

Log:

task
tools shown to model
tool descriptions
selected tool
arguments
validation result
execution result
next observation

Then classify the failure.

Wrong tool, but descriptions overlap

Fix the tool boundaries.

Add explicit use when and do not use when language.

Correct tool does not exist

The action space is incomplete.

Adding prompt instructions will not manufacture a capability.

Tool exists but is rarely selected

Check whether its description is dominated by a more generic tool.

Tool selection is correct but arguments fail

This is a schema problem, not a routing problem.

Tool executes but result is useless

Inspect the arguments and observation structure.

Agent keeps bouncing between two tools

Feed tool history and progress into the loop controller.

Do not debug every tool problem as a prompting problem.

Debugging: the agent has too many tools

Measure selection accuracy as the tool set grows.

For example:

5 tools
10 tools
20 tools
40 tools

If routing degrades, test:

better descriptions
dynamic subsets
hierarchical routing
tool families
specialist agents

Do not immediately jump to multi-agent architecture.

A small deterministic routing layer may solve the problem.

Debugging: the model ignores tool results

This is often an observation problem.

Check whether the result is:

structured
short enough
relevant
explicit about success/failure
linked to the action that produced it

A result like:

{
    "tool": "run_tests",
    "ok": False,
    "failed": 2,
    "failures": [...],
}

is easier to reason over than a large raw terminal dump.

You can still preserve the raw output for debugging.

The model does not necessarily need all of it on every step.

Do you need an LLM router?

Probably not at first.

Try this order:

1. sharpen tool boundaries
2. reduce overlapping tools
3. validate schemas
4. restrict tools by state
5. add deterministic routing where obvious
6. benchmark routing accuracy
7. only then consider a learned/LLM router

This follows the same rule as the rest of this series:

Add the smallest mechanism that fixes a measured failure.

An experiment worth running

Create 100 representative tasks.

Label the correct first tool for each.

Then compare:

A: original descriptions
B: explicit use / do-not-use descriptions
C: B + state-restricted tools
D: C + dedicated router

Measure:

first-tool accuracy
valid-argument rate
successful execution rate
verified task success
mean tool calls
latency
cost

The interesting result may be that B or C beats D.

A more complicated router is not automatically a better router.

The deeper lesson

It is tempting to think of agent tools as plugins attached to an intelligent model.

A better mental model is:

agent intelligence
      =
model policy
      +
action space
      +
observations
      +
runtime constraints

Change the action space and you change the problem the model is solving.

A model choosing among:

read
search
edit
test

has a much easier problem than a model choosing among dozens of overlapping functions with vague descriptions.

That is not prompt engineering trivia.

It is system architecture.

What we have built so far

The series now looks like this:

00  agent loop
 ↓
01  structured actions + validation
 ↓
02  multiple candidates + ranking
 ↓
03  critique + revision
 ↓
04  planning + execution
 ↓
05  state + progress + stopping
 ↓
06  tools + routing + schemas

Each mechanism adds something different.

And none of them is automatically required.

The next problem appears once the agent works across longer tasks or multiple runs:

What happens when the agent needs information it no longer has in the current context?

That is where memory begins.

Next: Agents From First Principles 07

AI Agent Forgets Previous Work? Add Working, Semantic and Episodic Memory

We will separate three ideas that are frequently collapsed into one word:

state
memory
retrieval

Then we will build each one independently and look at where they appear in coding agents, research systems, support software, long-running automation and personal assistants.