Chapter 20 assembled the book’s mechanisms into one loop. This appendix does something smaller and more specific: it shows three of those mechanisms already running in a production codebase that was not written for this book, and it holds itself to a stricter evidence rule than the chapters do.

Evidence rule for this appendix. There are two tiers here. The mechanics around each language-model call — UCT selection, the budget gate, the trace cache, the champion-update rule, the output parser — are deterministic software, and a committed fixture (experiments/dspy-from-first-principles/ch21_beyond_book/) exercises them with no LM and no DSPy import. Those results are measured. The kernels themselves — the production modules the excerpts are drawn from — are selection evidence from one codebase, not a generalisation. Their full source lives in the Stephanie repository, linked below and pinned to main at revision 32d02ee (2026-01-06).

Section	Full source
1. Reasoning search with a budget	stephanie/agents/dspy/mcts_reasoning.py
2. Memory that refuses to regress	stephanie/agents/dspy/memento.py
3. A small CBR module	stephanie/agents/dspy/cbr_dspy.py

The pack has more files than the three excerpted here. Three are in it but deliberately left out, for the same audit reason Chapter 17 applied to its own code: lats_dspy.py unpacks forward() into names it never returns and calls a non-existent example.with_score(...); learning_from_learning.py calls dspy.Advise, which is not a current optimizer and would fail exactly like the pre-fix MIPRO and GEPA snippets in Chapters 12 and 13; prompt_compiler.py is built on a non-DSPy LLMCompiler. Working code in a real repository is not automatically correct code, and a book excerpt should not launder it.

The three kernels map straight back onto the book. Section 1 is the production shape of Chapter 17’s reasoning search. Section 2 is Chapters 16 and 19 combined — memory plus promotion. Section 3 is Chapter 7’s “examples as data” at the smallest possible scale. None of them changes a conclusion reached in Chapters 1–20.

1. Reasoning search with a budget

Chapter 17 built a reasoning-search agent and stopped short of the cost boundary. This is the part it only described: every language-model call in the search passes through one gate.

The program shell separates generation from judgement — two signatures, two predictors:

class TraceStep(Signature):
    state = InputField(desc="Current problem state")
    trace = InputField(desc="History of thoughts/actions so far")
    next_step = OutputField(desc="Next reasoning step")

class ValueEstimator(Signature):
    state = InputField(desc="Current problem state")
    trace = InputField(desc="Reasoning steps taken")
    goal = InputField(desc="Goal text")
    score = OutputField(desc="Normalized score (0-1)")
    rationale = OutputField(desc="Why this path is promising")

class MCTSReasoningProgram(dspy.Module):
    def __init__(self, cfg):
        super().__init__()
        self.generator = Predict(TraceStep)
        self.value_estimator = Predict(ValueEstimator)
        self.max_depth = int(cfg.get("max_depth", 3))

These fields are untyped, so DSPy defaults them to strings. The book’s convention is name: type = InputField(...); the type annotations are documentation the compiler does not enforce, which is itself the Chapter 3 point about where a contract actually lives. The behaviour is identical either way.

Selection is standard UCT. An unvisited child returns infinity, which forces the search to try every branch once before the statistics mean anything:

def uct_value(self, parent_visits, ucb_weight):
    if self.visits == 0:
        return float("inf")
    exploitation = self.reward / max(1, self.visits)
    exploration = ucb_weight * math.sqrt(
        max(1e-9, math.log(max(1, parent_visits)) / self.visits)
    )
    return exploitation + exploration

The cost boundary is one method. Every generation call checks the budget first and consults a least-recently-used cache keyed on the tail of the trace:

def _predict_next(self, state: str, trace: list[str]) -> str:
    if self.calls_used >= self.max_lm_calls:
        return ""
    key = self._cache_key(state, trace)
    cached = self._cache_get(key)
    if cached is not None:
        return cached
    pred = self.generator(state=state, trace=trace)
    self.calls_used += 1
    nxt = pred.next_step.strip() if getattr(pred, "next_step", None) else ""
    self._cache_put(key, nxt)
    return nxt

With max_lm_calls=12, the search cannot spend its way to a good-looking accident. The cache has a second effect the book cares about: an identical (state, trace-tail) pair costs one call, so repeated structure in the search is paid for once — Chapter 8’s “name the baseline before optimising” applied to the search itself.

Evaluation is scheduled, not continuous. The eval_at switch (leaf, or every_k with eval_stride) decides which nodes are scored at all:

def _should_eval(self, node) -> bool:
    if self.eval_at == "leaf":
        return self._is_terminal(node)
    return (len(node.trace) % max(1, self.eval_stride)) == 0

This is the concrete form of Chapter 17’s warning about value-function legitimacy. Score every node and the search fits the scorer; score only leaves and most of the tree runs blind. There is no correct setting — only the one whose cost you have written down, which is why the whole configuration (max_depth, branching_factor, num_simulations, ucb_weight, max_lm_calls, eval_at, eval_stride) lives in one place at construction time.

The fixture confirms. Over a fixed table — an unvisited child, an exploited leader (10 visits, reward 7.0), and an under-explored child (3 visits, reward 0.9), parent visits 13, ucb_weight=1.41 — selection order is unvisited → under-explored → leader. The under-explored child scores 1.60 against the leader’s 1.41 despite a lower average reward (0.30 vs 0.70): the exploration term is doing real work, not decoration. The budget gate holds calls to the ceiling, and replaying an earlier (state, trace-tail) returns the cached step at zero additional cost.

2. Memory that refuses to regress

Chapter 16 built retrieval and memory; Chapter 19 built promotion with rollback. The production system that joins them is a case-based reasoning agent with one structural guarantee: the champion is replaced only by something measured better, under the same seed.

Each scheduled run compares two variants. A is the baseline with no casebook and retention off. B is the full case-based path with retention on. The seed is locked so the delta measures the casebook, not sampling noise:

base_res = await self._run_single_variant(
    context, variant="baseline",
    use_casebook=False, retain=False,      # control never writes to memory
    casebook_tag=self.ab_control_tag,
)
cbr_res = await self._run_single_variant(
    context, variant="cbr",
    use_casebook=True, retain=True,
    casebook_tag=self.casebook_tag,
)

q_base = float(base_res["metrics"]["quality"])
q_cbr = float(cbr_res["metrics"]["quality"])
improved = q_cbr > (q_base + self.ab_delta_eps)

self._update_non_regression(home_casebook_id, goal["id"], cbr_res, improved)

winner_variant = "cbr" if improved else "baseline"
context[self.output_key] = context.get(self._variant_output_key(winner_variant), [])

Three details carry the book’s lessons. retain=False on the baseline arm means the control cannot write to the memory it anchors — Chapter 18’s firewall as code. The two variants write to namespaced output keys and only the winner is projected to the canonical output, so a losing candidate cannot leak downstream by accident — Chapter 19’s separation of promotion from activation, enforced by key structure. And ab_delta_eps is only a floor, not a margin: a fourth-decimal difference can clear it while meaning nothing.

The promotion itself does nothing on the failing case, which is the whole guarantee:

def _update_non_regression(self, casebook_id, goal_id, run_result, improved):
    """Promote the champion only on a non-regressing improvement."""
    if not improved:
        return
    ...
    self.memory.casebooks.upsert_goal_state(casebook_id, goal_id, case_id, quality)

No demonstration, no reflection, no trajectory is promoted on potential. This is the production form of the sentence Chapter 14 repeats — reflection proposes, evaluation disposes — with the write path held closed unless the number clears.

The reuse path that feeds the CBR arm follows a fixed priority (champion, then recent successes, then diverse novel cases) capped at a reuse budget, so retrieval is a policy with a budget, not an open tap. The full 1,027-line file — training-event plumbing, micro-learning hooks, namespaced scratch contexts — is behind the source link; the kernel above is the part the argument needs.

The fixture confirms. Over five quality pairs against a champion at 0.70: a tie holds the champion; a +0.0000005 difference (below ab_delta_eps = 1e-6) holds it; +0.0005 clears the epsilon and promotes, but fails a practical 0.01 margin — the fixture flags exactly this gap; +0.08 promotes cleanly; and a -0.09 regression holds. Final champion quality 0.78. The epsilon stops zero from promoting; only a real margin stops noise from promoting.

3. A small CBR module that earns its place

After the two large systems, the most instructive file in the pack is one of the smallest: a signature, a module, and a parser — and the parser is where the lesson lives.

The contract separates what the program consumes from what downstream software may rely on, exactly as Chapter 3 prescribes:

class CBRSignature(Signature):
    """DSPy signature for CBR hypothesis generation."""
    documents: str = InputField(desc="Relevant documents or text passages to review")
    goal: str = InputField(desc="The research or reasoning goal")
    task_instructions: str = InputField(desc="Task framing and example for CBR")
    hypotheses: list[str] = OutputField(desc="Clear, testable, actionable hypotheses")

class CBRModule(dspy.Module):
    def __init__(self):
        super().__init__()
        self.generator = dspy.Predict(CBRSignature)

    def forward(self, documents, goal, task_instructions=""):
        return self.generator(
            documents=documents, goal=goal, task_instructions=task_instructions,
        )

task_instructions is the field to notice. It carries the task framing and a worked example into the call as data, not as hidden promptcraft — which makes the instruction a versioned input: visible, diffable, freezable, rather than part of the program’s identity. When Chapter 12’s instruction search rewrites wording, this is the status of the thing it is rewriting.

The output declares list[str] and then declines to trust the declaration. Model outputs arrive as lists, as numbered text blocks, or as something else, so the module normalises all three and caps the result:

def _parse_hypotheses(self, raw) -> list[str]:
    hypotheses = []
    if isinstance(raw, list):
        for h in raw:
            if isinstance(h, str) and len(h.strip()) > 10:
                hypotheses.append(h.strip())
    elif isinstance(raw, str):
        for line in raw.strip().split("\n"):
            line = re.sub(r"^\d+[\.\)]\s*", "", line).strip()
            if len(line) > 10:
                hypotheses.append(line)
    else:
        self.logger.log("ParseHypothesesError",
                        {"type": str(type(raw)), "value": str(raw)[:200]})
    return hypotheses[: self.max_hypotheses]

This is Chapter 5’s “explicit stages must earn causal influence” at the smallest scale: the declared type does not parse, the parser parses, and the third case — the shape nobody predicted — is logged with its type and a 200-character sample rather than silently dropped. The 4,000-character document truncation elsewhere in the module is the same discipline applied to inputs: a visibility boundary with a log line, so context-window pressure is observed rather than absorbed.

The pack’s chain-of-thought file gets no excerpt: its signature (question, optional references and preferences, to answer) is exactly Chapter 4’s pattern — one contract, reasoning style carried as an input — and the book already teaches it.

The fixture confirms. The list input [" If steps are stored as cases… ", "short", 42] normalises to one hypothesis (whitespace trimmed, the too-short string and the integer dropped). The numbered-text block normalises to two (the "3. x" line dropped as too short). The dictionary input returns [] with shape unexpected_shape_logged — recorded, not discarded.

4. Simulating Recurrent Reasoning Depth

The previous sections looked backward: three mechanisms already present in production code. We will end by looking forward.

Recent recurrent-depth architectures suggest another way to spend inference-time computation. Instead of generating a longer visible reasoning trace or searching several alternative reasoning paths, a model can repeatedly transform an evolving internal representation before producing its answer.

DSPy cannot reproduce that mechanism directly. A DSPy module sees language-model inputs and outputs; it does not ordinarily expose the Transformer’s hidden state or let us send that hidden state through the same neural block again.

But we can reproduce the computational shape and make its most important variable explicit:

problem
   ↓
state₀
   ↓
same transformation
   ↓
state₁
   ↓
same transformation
   ↓
state₂
   ↓
same transformation
   ↓
state₃
   ↓
answer

Instead of searching across several reasoning paths, we repeatedly deepen one state.

That gives us a new experimental variable:

recurrent depth = 1, 2, 4, 8, ...

The smallest DSPy version requires two contracts. One transforms the state. The other converts the final state into an answer.

import dspy


class RefineState(dspy.Signature):
    """Update the current problem representation.

    Do not produce the final answer. Preserve useful information,
    correct weak assumptions, and make the state more useful for
    solving the problem.
    """

    problem: str = dspy.InputField()
    state: str = dspy.InputField(
        desc="Current compact problem representation"
    )

    next_state: str = dspy.OutputField(
        desc="Revised compact problem representation"
    )


class SolveFromState(dspy.Signature):
    """Produce the answer from the final problem representation."""

    problem: str = dspy.InputField()
    state: str = dspy.InputField()

    answer: str = dspy.OutputField()

The recurrent program is ordinary Python around two DSPy modules:

class RecurrentReasoner(dspy.Module):
    def __init__(self, depth: int):
        super().__init__()
        self.depth = depth
        self.refine = dspy.Predict(RefineState)
        self.solve = dspy.Predict(SolveFromState)

    def forward(self, problem: str):
        state = problem

        states = []

        for step in range(self.depth):
            result = self.refine(
                problem=problem,
                state=state,
            )

            state = result.next_state
            states.append(state)

        result = self.solve(
            problem=problem,
            state=state,
        )

        return dspy.Prediction(
            answer=result.answer,
            states=states,
            depth=self.depth,
        )

Nothing about the model changes between iterations.

same LM
same signature
same module
same task
same initial problem

Only the amount of inference-time computation changes.

That lets us compare:

depths = [0, 1, 2, 4, 8]

for depth in depths:
    program = RecurrentReasoner(depth=depth)
    result = program(problem=problem)

    evaluate(
        answer=result.answer,
        depth=depth,
    )

depth=0 is especially important. It is the control condition: synthesize directly from the original state without recurrent refinement.

Now depth itself has become an experimental variable.

Three Ways to Spend Reasoning Compute

This gives us a useful contrast with the reasoning systems built earlier in the book.

A chain-of-thought program spends additional computation by extending a sequence:

thought₁
   ↓
thought₂
   ↓
thought₃
   ↓
answer

Chapter 17’s tree search spends it by exploring alternatives:

                 state
              /    |    \
             /     |     \
            A      B      C
           / \           / \

The recurrent simulation spends it by repeatedly transforming one state:

state₀
   ↓
state₁
   ↓
state₂
   ↓
state₃
   ↓
answer

Those are three different compute-allocation policies:

    flowchart LR
    C[additional inference compute] --> LT[longer trajectory: ChainOfThought]
    C --> WS[wider search: Best-of-N, tree search, MCTS]
    C --> DS[deeper state transformation: recurrent refinement]
  

The distinction matters because equal compute budgets need not produce equal behavior.

The Experiment

A useful first comparison would hold the task, model, dataset, metric, sampling configuration and evaluation policy fixed while varying only the reasoning strategy.

Direct

ChainOfThought

Recurrent depth = 1

Recurrent depth = 2

Recurrent depth = 4

Recurrent depth = 8

For each condition record at least:

task score
hard regressions
LM calls
input tokens
output tokens
wall-clock latency
cost

For recurrent conditions, also retain every intermediate state.

That gives us another useful measurement:

depth       score       marginal gain

0           ...
1           ...
2           ...
4           ...
8           ...

The interesting question is not simply whether depth eight scores highest.

It is:

What does another unit of inference-time computation buy?

The possible outcomes all teach us something.

score rises with depth
    → repeated refinement appears useful

score rises then plateaus
    → the task has a useful reasoning-depth frontier

score rises then falls
    → more computation produces overthinking or state drift

score does not move
    → the recurrence is paying for repeated reformulation

cost rises faster than quality
    → recurrence works but does not earn its budget

A particularly important failure mode is state collapse.

Repeated refinement could gradually remove information:

state₀
    all relevant evidence

state₁
    useful abstraction

state₂
    shorter abstraction

state₃
    confident summary

state₄
    missing the qualifier that mattered

More reasoning would then make the representation worse.

That possibility connects recurrent depth directly to the central argument of this book: additional optimization or computation is not synonymous with improvement.

Where the Analogy Stops

This experiment must not be described as implementing latent recurrent reasoning.

A real recurrent-depth model can approximately perform:

hₜ₊₁ = R(hₜ)

where h is a continuous neural representation inside the network.

Our DSPy program performs something closer to:

textₜ₊₁ = LM(problem, textₜ)

The state therefore crosses a language bottleneck on every iteration.

That difference is substantial.

The DSPy experiment cannot tell us whether reasoning directly in hidden representations is faster, richer, less interpretable, or more capable than reasoning through tokens.

What it can test is the architectural proposition underneath the technique:

Can a fixed language-model program benefit when the same transformation is applied repeatedly to an evolving state, and how does that benefit change as inference depth increases?

That is enough to make recurrence an experimental object rather than an architectural mystery.

And it gives us one final extension of the book’s original progression:

prompt wording
    ↓
program

program
    ↓
measurable behaviour

behaviour
    ↓
optimization

optimization
    ↓
search over programs

agents
    ↓
search over actions

retrieval
    ↓
search over evidence

reasoning search
    ↓
search over reasoning paths

recurrent reasoning
    ↓
search over computational depth

DSPy cannot reach inside the Transformer and perform latent recurrence.

But it can let us ask the engineering question first:

If thinking longer no longer has to mean writing more thoughts, where should the extra computation go?

What Usually Goes Wrong

The failures cluster where the book says they will — at the boundaries, not inside the model calls.

Conclusion

This appendix changes no conclusion in Chapters 1–20. It shows the book’s machinery running one level up in code that predates the book: a search with a written-down budget, a memory whose champion moves only on a margin-clearing improvement, and a small module whose parser does the work its type signature merely declares.

The pattern across all three kernels is the one the book has repeated since Chapter 2. The language model generates. Deterministic software — budgets, caches, seed locks, namespaced keys, parsers, margins — decides what that generation is allowed to mean. Production did not need a new idea. It needed the old ideas to have write paths that stay closed by default.

The mechanics above are reproduced by the committed fixture; the LM-backed behaviour of the full agents is not run here, and their numbers would be selection evidence from one codebase in any case. Run them against your own tasks before trusting the shapes they produce.

Exercises

Both run against experiments/dspy-from-first-principles/ch21_beyond_book/.

1. Force a wrong cache hit. In run_budget_cache, change the replay tail from trace[:2] to ["unrelated step", "draft patch"]. With tail_len=1 the key is just ("draft patch",) — a key the forward walk already stored — so the cache returns a step generated for a different reasoning history at zero cost. Set tail_len=6 and confirm the same replay is now a miss. Which is the more dangerous failure in production: paying for a repeat you could have cached, or reusing a step that does not belong to this path?
2. Distrust the margin. The A/B fixture flags the third pair (0.7005 vs 0.70) with clears_margin_0_01: false even though it promotes under delta_eps. Lower margin_demo until that flag flips to true; you will land on 0.0005. Now the epsilon and the margin — which the code treats as different concepts — are the same number for this pair. What is ab_delta_eps actually protecting against, if only a real margin protects against noise?

Further Reading

• Case-Based Reasoning: Foundational Issues (Aamodt and Plaza, 1994): the retrieve–reuse–revise–retain cycle that Section 2’s casebook and Section 3’s module both instantiate. (AI Communications 7(1))
• The Tail at Scale (Dean and Barroso, 2013): why a hard per-call budget that returns a degraded result beats an unbounded one that returns a great result late. (CACM 56(2))
• Monte Carlo Tree Search: A Review (Browne et al., 2012): the UCT rule in Section 1, in full. (IEEE)