Example

Validate the whole proposal before any side effect

Run a proposal through the action boundary and see the schema, semantic and authorization failures kept apart, with the executor only ever receiving a typed action.

A model can propose more than a runtime should do, and the chapter’s claim is that the boundary has to answer three separate questions in order, and fail loudly at each. Does the proposal have the right shape? Are the values usable? And may this run perform it? A proposal that passes the first two can still be refused by the third.

Run it

The boundary is the chapter’s own code. Save the first block as action_boundary.py, save the second as demo.py beside it, and run the driver with Python 3.11 or later. The standard library only: no model, no network, no service.

Expected outcome

a well-formed search
  accepted: SearchAction -> ['result 0 for retrieval contract', 'result 1 for retrieval contract', 'result 2 for retrieval contract']
an unknown action
  refused: SchemaError: Unknown action: publish
an invented extra argument
  refused: SchemaError: Unexpected fields in search arguments: ['depth']
a missing argument
  refused: SchemaError: Missing fields in search arguments: ['max_results']
a well-formed but empty query
  refused: SemanticError: query cannot be empty
a well-formed but out-of-range count
  refused: SemanticError: max_results must be between 1 and 20
a well-formed final answer, not allowed in this run
  refused: PermissionError: Action not allowed in this run: final

tool calls made: ["search('retrieval contract', max_results=3)"]
the executor only ever received a typed action: True

One tool call was made, for the one proposal that cleared every rung. The final action is well formed: it has the right keys and usable values, and it is still refused, because the run’s policy does not permit it.

The boundary

The chapter’s code, in full, with two validators added in the shape the chapter specifies for them. The action types, the two exception types, the exact-key check, the search validator, the authorisation step and the executor are the chapter’s own; validate_calculate and validate_final are written here in the shape the chapter describes for them.

"""The action boundary from the agents-from-first-principles chapter 2."""

from dataclasses import dataclass
from enum import StrEnum
from typing import Literal


class Stage(StrEnum):
    REPRESENTATION = "representation"
    SCHEMA = "schema"
    SEMANTICS = "semantics"
    AUTHORIZATION = "authorization"
    PRECONDITION = "precondition"
    EXECUTION = "execution"


@dataclass(frozen=True)
class SearchAction:
    kind: Literal["search"]
    query: str
    max_results: int


@dataclass(frozen=True)
class CalculateAction:
    kind: Literal["calculate"]
    expression: str


@dataclass(frozen=True)
class FinalAction:
    kind: Literal["final"]
    answer: str


Action = SearchAction | CalculateAction | FinalAction


class SchemaError(ValueError):
    """Structure, keys or primitive types are wrong."""


class SemanticError(ValueError):
    """Structure is correct; the values are not usable."""


def require_exact_keys(data: dict, expected: set[str], *, where: str) -> None:
    actual = set(data)
    missing = expected - actual
    extra = actual - expected

    if missing:
        raise SchemaError(f"Missing fields in {where}: {sorted(missing)}")

    if extra:
        raise SchemaError(f"Unexpected fields in {where}: {sorted(extra)}")


def validate_search(args: dict) -> SearchAction:
    require_exact_keys(args, {"query", "max_results"}, where="search arguments")

    query, max_results = args["query"], args["max_results"]

    if not isinstance(query, str):
        raise SchemaError("query must be a string")

    if type(max_results) is not int:
        raise SchemaError("max_results must be an integer")

    query = query.strip()

    if not query:
        raise SemanticError("query cannot be empty")

    if not 1 <= max_results <= 20:
        raise SemanticError("max_results must be between 1 and 20")

    return SearchAction("search", query, max_results)


def validate_calculate(args: dict) -> CalculateAction:
    require_exact_keys(args, {"expression"}, where="calculate arguments")

    expression = args["expression"]

    if not isinstance(expression, str):
        raise SchemaError("expression must be a string")

    expression = expression.strip()

    if not expression:
        raise SemanticError("expression cannot be empty")

    return CalculateAction("calculate", expression)


def validate_final(args: dict) -> FinalAction:
    require_exact_keys(args, {"answer"}, where="final arguments")

    answer = args["answer"]

    if not isinstance(answer, str):
        raise SchemaError("answer must be a string")

    return FinalAction("final", answer.strip())


def validate_proposal(data: object) -> Action:
    if not isinstance(data, dict):
        raise SchemaError("Action proposal must be an object")

    require_exact_keys(data, {"action", "arguments"}, where="action proposal")

    name, arguments = data["action"], data["arguments"]

    if not isinstance(name, str):
        raise SchemaError("action must be a string")

    if not isinstance(arguments, dict):
        raise SchemaError("arguments must be an object")

    validators = {
        "search": validate_search,
        "calculate": validate_calculate,
        "final": validate_final,
    }

    if name not in validators:
        raise SchemaError(f"Unknown action: {name}")

    return validators[name](arguments)


@dataclass(frozen=True)
class RunPolicy:
    allowed_actions: frozenset[str]


def authorize(action: Action, policy: RunPolicy) -> None:
    if action.kind not in policy.allowed_actions:
        raise PermissionError(f"Action not allowed in this run: {action.kind}")


def execute(action: Action, tools):
    if isinstance(action, SearchAction):
        return tools.search(query=action.query, max_results=action.max_results)

    if isinstance(action, CalculateAction):
        return tools.calculate(action.expression)

    if isinstance(action, FinalAction):
        return action.answer

    raise TypeError(f"Unsupported validated action: {type(action)}")

The driver

The driver is not part of the chapter. It runs seven proposals through the boundary and prints what happened to each.

"""Driver for the agents-from-first-principles chapter 2 action boundary."""

from action_boundary import (
    Action,
    FinalAction,
    RunPolicy,
    SchemaError,
    SearchAction,
    SemanticError,
    authorize,
    execute,
    validate_proposal,
)


class Tools:
    def __init__(self) -> None:
        self.calls: list[str] = []

    def search(self, *, query: str, max_results: int):
        self.calls.append(f"search({query!r}, max_results={max_results})")
        return [f"result {index} for {query}" for index in range(max_results)]

    def calculate(self, expression: str):
        self.calls.append(f"calculate({expression!r})")
        return f"value of {expression}"


tools = Tools()
policy = RunPolicy(allowed_actions=frozenset({"search", "calculate"}))


def attempt(label, proposal):
    try:
        action = validate_proposal(proposal)
    except (SchemaError, SemanticError) as error:
        print(f"{label}\n  refused: {type(error).__name__}: {error}")
        return
    try:
        authorize(action, policy)
    except PermissionError as error:
        print(f"{label}\n  refused: PermissionError: {error}")
        return
    result = execute(action, tools)
    print(f"{label}\n  accepted: {type(action).__name__} -> {result!r}")


attempt(
    "a well-formed search",
    {"action": "search", "arguments": {"query": "retrieval contract", "max_results": 3}},
)
attempt(
    "an unknown action",
    {"action": "publish", "arguments": {"answer": "done"}},
)
attempt(
    "an invented extra argument",
    {"action": "search", "arguments": {"query": "x", "max_results": 3, "depth": 9}},
)
attempt(
    "a missing argument",
    {"action": "search", "arguments": {"query": "x"}},
)
attempt(
    "a well-formed but empty query",
    {"action": "search", "arguments": {"query": "   ", "max_results": 3}},
)
attempt(
    "a well-formed but out-of-range count",
    {"action": "search", "arguments": {"query": "x", "max_results": 500}},
)
attempt(
    "a well-formed final answer, not allowed in this run",
    {"action": "final", "arguments": {"answer": "the answer"}},
)

print()
print("tool calls made:", tools.calls)
print("the executor only ever received a typed action:", all(
    isinstance(a, (SearchAction, FinalAction)) for a in [validate_proposal(
        {"action": "search", "arguments": {"query": "x", "max_results": 1}})]
))

Mechanism and limitations

Documented: the stage vocabulary, the typed actions, the two exception types, the exact-key rule, the search validator, the authorisation step and the executor are in the chapter’s own text, and the code above is that text’s code. The chapter also names the failure it is preparing for: a half-interpreted proposal that has already begun writing files leaves the system in a state no rung of the ladder approved.

Observed: the run above. An unknown action and an invented extra argument were refused as schema failures; an empty query and an out-of-range count were refused as semantic failures; a well-formed final answer was refused by authorisation; and the executor was called once, with a SearchAction.

Not shown. The chapter’s validate_calculate and validate_final follow the same shape as the search validator and are written here in that shape; the chapter states the pattern rather than showing those two in full. The boundary here stops at authorisation: the chapter’s precondition rung and its repair policy are not in this program, so a refused proposal simply stops. The tools are a stub. Nothing here shows that a model produces better proposals because the boundary is sharper, and nothing here shows that a refusal is understood.

Read the chapter .

Understand this example

Copy this prompt into your AI tool. No code runs here.

Apply this example

Copy this prompt into your AI tool. No code runs here.