Compile the Program

Chapter 9 gave the optimizer something to pursue: a metric. That does not mean we should let an optimizer mutate the accepted program.

Compilation in DSPy is the process of taking a program, optimizer-visible evidence, a metric, and an optimizer, then producing a candidate program state.

1. Baseline and candidate are different objects

Before compiling, name the boundary:

A candidate is not the new production program. It is a proposed program version that must be evaluated.

baseline v0.1
     ↓
compile on allowed data
     ↓
candidate v0.2
     ↓
evaluate untouched cases
     ↓
accept, reject, or keep investigating

CoCoder’s EngineeringProgram docs make this explicit: ProgramRuntime executes one concrete program version; ProgramOptimizer proposes a new candidate version. DSPy types are isolated inside the optimizer adapter. Holdout cases are persisted on the optimization run but not passed into the optimizer.

2. The minimal compile shape

Current DSPy optimizers expose a compile(...) pattern. For BootstrapFewShot:

Other optimizers use a validation set. For example, current MIPROv2 accepts valset during compile and uses it while searching candidate instruction and demonstration combinations:

optimizer = dspy.MIPROv2(
    metric=editorial_metric,
    auto="light",
    seed=13,
)
candidate = optimizer.compile(
    student=baseline,
    trainset=trainset,
    valset=devset,
)

That distinction matters. Current BootstrapFewShot.compile accepts student, optional teacher, and trainset; it does not take valset. Current MIPROv2.compile accepts both trainset and optional valset.

So dev is an experimental role, not a universal DSPy argument:

3. Audit before and after compile

Compilation should leave a trail. At minimum, record the state of the baseline before optimization, then record the candidate after optimization.

For DSPy modules, dump_state() serializes the state of named parameters. After compilation, that can expose predictor state such as demonstrations and optimized signature/instruction content.

It does not by itself describe the whole executable system. Python architecture, external tools, runtime configuration, LM/provider settings, dataset identity, and optimizer configuration belong in the surrounding manifest.

Current BootstrapFewShot also creates a reset copy of the student before training it, so the returned compiled student is distinct from the baseline state. Recording baseline_before and baseline_after still gives us a useful invariant: compilation must not silently change the accepted baseline object.

The important habit is to never let compile be an unaudited transition.

4. Persist the candidate state

DSPy supports two importantly different persistence modes.

A .json save is state-only:

import hashlib
import json
from pathlib import Path

def write_manifest(path: str, payload: dict) -> str:
    blob = json.dumps(payload, indent=2, sort_keys=True)
    fingerprint = hashlib.sha256(blob.encode("utf-8")).hexdigest()
    manifest = {**payload, "manifest_fingerprint": fingerprint}
    Path(path).write_text(json.dumps(manifest, indent=2, sort_keys=True))
    return fingerprint

The manifest should answer:

5. CoCoder’s stricter protocol

CoCoder’s frozen DSPy experiment protocol is the larger version of this chapter:

freeze manifest
check corpus
evaluate baseline on holdout
run one DSPy optimization on train/development only
persist candidate program
audit candidate integrity
evaluate candidate on same holdout
compare with conservative policy
persist result
stop without promotion

There is a subtle holdout point in this protocol. CoCoder evaluates the baseline on the frozen holdout before optimization, then evaluates the candidate on the same holdout. The holdout remains valid because it is not passed to the optimizer and the frozen protocol does not tune prompts, policy, validation, dataset membership, or model configuration after observing those results.

So “untouched holdout” should mean not used to shape the candidate or protocol, not necessarily “never executed before candidate generation.” If a human inspected baseline holdout failures and then changed the metric or program before compilation, that holdout would have become development evidence.

The central rule is simple:

The optimizer proposes. It does not promote itself.

DSPy can generate a candidate and report optimizer-facing scores. Those scores are evidence about the optimization procedure, not permission to deploy. Promotion belongs to an independent comparison and policy layer.

For the editorial program, the same rule applies. A compiled sentence improver is a candidate. It must still be compared with the frozen baseline under the predeclared evaluation protocol and inspected for failure distribution.

6. What compilation is allowed to change

Before running an optimizer, decide which parts of the system are allowed to change.

The point is not that every DSPy optimizer changes the same fields. The manifest should declare the optimizer’s allowed mutation surface before the run. Anything outside that surface changing creates a different experiment.

What Usually Goes Wrong

Conclusion

We gained the compilation boundary. DSPy compilation is now an auditable transition from frozen baseline state to candidate state under an explicit optimizer visibility and mutation policy.

We removed two assumptions: that compile() is simply “better prompting,” and that an optimizer’s returned program or optimizer score should automatically replace the accepted version.

The next question is narrower: BootstrapFewShot changes program behavior by constructing demonstrations. Where do those demonstrations come from, which traces are admitted, and why should we trust them?