Agents Are Programs Too
Agents Are Programs Too
Chapter 13 gave failures language. The optimizer can receive feedback, reflect, and propose better instructions.
The next limitation is access. Our editorial program receives a sentence and local context. A repository repair program cannot be handed the whole world as one input and expected to behave.
The naive shape is:
issue
+
huge repository context
↓
model
↓
patch
The stronger shape gives the program bounded capabilities rather than one giant input:
1. A minimal repository tool boundary
Start with safe teaching tools. Read-only tools come first.
The tools are boring on purpose. More importantly, the repository root is bound by ordinary application code when RepositoryTools is constructed. It is not an LM-visible tool argument.
That is a real security boundary. If root were exposed as a tool parameter, the model could simply ask the tool to inspect another directory and redefine its own sandbox. Path resolution inside a model-supplied root would not protect anything.
The tool layer should also record the repository revision, tool version, query, and a fingerprint of returned evidence when those observations matter to an experiment. Mutation belongs behind a different capability boundary.
2. Current DSPy tool-use mechanisms
Current DSPy documents dspy.ReAct(signature, tools, max_iters=...) as a generalized tool-using module over any signature. Ordinary typed Python callables are wrapped as DSPy tools, and DSPy also provides dspy.Tool plus MCP conversion helpers.
Current ReAct runs an explicit reasoning/action loop and returns the declared outputs together with a trajectory dictionary containing thoughts, tool names, arguments, and observations.
DSPy 3.3 also introduced dspy.ReActV2 as an experimental native-tool-aware replacement. Current documentation describes it as using structured dspy.History, supporting multiple native tool calls in one turn, and eventually moving behind the canonical dspy.ReAct name. That is an evolving API surface, so pin the DSPy version and normalize agent traces behind your own application boundary rather than persisting assumptions about one internal representation.
The stable concept is:
signature
+
tool set
+
bounded reasoning/action loop
↓
structured output
Example shape:
The important test target is not only the LM call. The repository tools are ordinary Python and should be unit-tested against a fixture repository for root escape, symlinks, line bounds, result ordering, decoding failures, and output limits before an agent can use them.
There is another current ReAct behavior worth making explicit: tool exceptions are caught and converted into observation text so the loop can continue. A completed agent call therefore does not prove that every tool invocation succeeded. Evaluation should record tool errors separately and decide whether a run containing them is valid evidence.
3. The trajectory is an audit record
An agent produces more than a final answer. Current ReAct exposes a trajectory field precisely so we can inspect what happened during the reasoning/action loop:
state
↓
tool choice
↓
tool input
↓
observation
↓
next action
↓
final output
The trajectory helps answer execution questions:
A trajectory is evidence about execution, not proof that the final answer or patch is correct. A plausible sequence of searches can still lead to the wrong diagnosis.
Repository-repair outcome evidence belongs outside the reasoning trace: materialize the candidate in an isolated workspace, check that the patch applies, run the declared validation, verify scope, and inspect regressions. The trajectory explains how the candidate was reached; external validation establishes what the candidate actually does.
The environment is part of that evidence. Tool observations should be tied to the repository revision and tool configuration that produced them, otherwise a later rerun may inspect different code and appear to tell the same story.
4. Optimize outcome, not performance theater
A repository-repair metric should prioritize verified outcome:
What Usually Goes Wrong
| Symptom | Likely cause | How to diagnose it | What to change |
|---|---|---|---|
| Agent can inspect outside the repository | Repository root is an LM-visible tool argument | Inspect generated tool schema and attempted paths | Bind root/capabilities outside the model-visible tool interface |
| Agent loops | Tool results do not create a useful stopping signal | Inspect repeated tool names, arguments, and observations | Keep a hard max_iters cap; improve tool contracts and task/finish criteria |
| Agent returns an answer after a tool failed | Current ReAct converted the exception into an observation | Inspect trajectory observations for execution errors | Record structured tool status and fail or penalize invalid runs |
| Tool calls look sensible but answer is wrong | Metric rewards trajectory rather than verified outcome | Compare trace features with external validation | Score the validated result; keep trajectory metrics secondary |
| Agent uses mutation during diagnosis | Capability boundary is too broad | Review the phase-specific tool list | Separate read-only inspection, mutation, and validation capabilities |
| Tool output overwhelms model | Tool returns too much text | Record observation sizes and truncation | Bound lines, hits, fields, and total observation budget |
| Same issue gives different evidence | Repository state, result ordering, or tool config changed | Compare revision/tool fingerprints and ordered results | Freeze revision/config; sort and fingerprint tool outputs |
| Trace parser breaks after a DSPy upgrade | Application depended on ReAct/ReActV2 internals | Compare DSPy version and returned trajectory/history shape | Pin DSPy and normalize traces into an application-owned DTO |
Conclusion
We gained bounded environment access without giving up the program abstraction. An agent is still a DSPy program with a task contract, a capability set, a bounded loop, an execution record, and an external evaluation protocol.
We removed two assumptions: that tool use sits outside the optimization story, and that an agent’s plausible trajectory is evidence that its final result is correct.
Tools solve access to an environment, but they do not solve scale. A repository can contain thousands of files. A book can contain millions of tokens. The next problem is context.
