Tool Choice Is a Behavioral Problem

A tool can be perfectly implemented and never used correctly.

Once an agent sees several capabilities, success depends on a behavioral pipeline:

discover → select → construct arguments → invoke → interpret result

A single “task passed” score cannot tell us which stage failed.

1. Discovery precedes choice

If the tool never enters the agent’s effective context, it cannot be selected. Record which registrations were available, filtered, truncated or rejected.

Tool-list pressure matters. An agent choosing among five operations faces a different problem from one choosing among five hundred.

2. Names and descriptions change behavior

Compare:

search_book — Search this book for relevant passages.

with:

find — Gets useful information.

Both can call identical code. Their discovery behavior will differ.

Descriptions should distinguish tools rather than advertise them. Overlapping descriptions create routing ambiguity; persuasive descriptions create a security problem.

3. Evaluate negative cases

A tool-choice suite needs cases where the correct behavior is:

• choose search_book;
• choose get_chapter;
• ask for clarification;
• use no tool;
• refuse an unauthorized tool;
• stop after an empty result.

If every fixture requires a tool, the evaluation rewards unnecessary action.

4. Score stages independently

One trace can yield:

An agent might choose the right tool with the wrong chapter number. Another might invoke it correctly and then ignore its result.

These require different fixes.

5. Similar tools expose boundaries

Introduce controlled competitors:

search_book(query)
search_current_chapter(query)
search_site(query)

The goal is not to trick the model. It is to discover whether our capability boundaries are intelligible.

Persistent confusion may mean the tools should be merged, renamed or given non-overlapping scopes.

Evaluation can improve the interface rather than only the model prompt.

6. Arguments need semantic checks

Schema-valid arguments can still be wrong:

{
  "query": "context windows",
  "limit": 10
}

Ten is valid but wasteful when one result is enough. A chapter identifier may parse while naming the wrong book.

Argument evaluation should protect relevance, scope, cost and authority—not just shape.

7. Tool results are untrusted inputs

The result may contain stale data, application errors or malicious text. The agent must distinguish content returned by a tool from instructions governing the agent.

A grounded final answer should cite or identify the result it used. If the tool returns nothing, the model should not fill the gap from imagination while implying the search succeeded.

8. Compare behavior across configurations

The local coordinator can send the same tool-choice job to multiple Chrome workers. Each result carries worker provenance, discovered tool set and decision trace.

Paired cases let us ask whether a browser/model change altered:

• unnecessary tool use;
• correct selection;
• invalid arguments;
• abstention;
• result grounding;
• latency and context cost.

This is more informative than comparing final prose alone.

Conclusion

Tool choice is not one decision. It is discovery, selection, argument construction, admission, execution and result use.

The Observatory must preserve every stage. The next chapter adds hostile content and executable authority, where a mistaken stage can become a security failure rather than a poor answer.

Sources and further reading

1. Chrome for Developers, WebMCP.
2. Model Context Protocol, Specification.