Tool Calling and Agentic Inference

Generation, interruption, tool handoff, re-entry.

Inference From First Principles Chapter 34 of 45

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

Chapter 33 examined reasoning budgets. This chapter examines tool use.

Tool calling

Tool calling lets a model request external computation. The runtime must parse the request, execute safely, and return results. The model proposes calls; the application is responsible for authorisation, execution, validation, and retries.

Interruption and re-entry

Generation pauses for tool execution, then resumes with the tool result. This requires state management and trace IDs.

Sandboxing

Tools must be sandboxed. Untrusted model output must not execute arbitrary code. Tool arguments and results are untrusted data requiring schemas and policy checks.

Research context

OpenAI and Anthropic documentation frame tool use as a proposal/execution loop: the model emits a structured call, the application validates and executes it, then returns a tool result. Harmony adds a protocol-level representation for compatible models. The application remains responsible for safety; the protocol does not make tool output trusted.

Experiment

The repository example implements a toy calculator tool with trace IDs and retries. Results were 4, 42, and error. Tests verify calculation, trace IDs, unknown tools, and serialisation.

Limitations

Toy calculator.
No real tool execution.
No end-to-end task measurement.
No security audit.

Key terms

Tool calling: requesting external computation.
Trace ID: unique request identifier.
Sandbox: restricted execution environment.
Re-entry: resuming generation.
Retry: repeated attempt.

Reader exercises

Add a file-read tool with allowlist.
Implement retry backoff.
Measure end-to-end task completion.
Explain risks of unsandboxed tools.

References

OpenAI function calling guide - official provider documentation.
Tool use with Claude - official provider documentation.
openai/harmony - first-party protocol repository.

Implementation status

Manuscript: content/books/inference/34-chapter.md
Runnable example: examples/ch34/
Tests: tests/test_ch34_tool.py
Evidence: research/ch34/sources.md and research/ch34/results.json
Verification command: python -m unittest discover -s tests -v

The chapter is a draft. It does not claim production tool safety. The next chapter will examine structured outputs.