Separate What From How

In chapter 3 we designed a stronger contract for sentence improvement. The signature names the inputs and outputs. It says what the LM should consume and produce.

It still does not say how the LM should execute the task.

That distinction is the main subject of this chapter.

                SAME CONTRACT

input ────────> Signature ────────> output
                    │
             execution policy
              /           \
         Predict       ChainOfThought

Changing the task contract and changing the execution strategy are different operations. Treating them as the same operation is one reason prompt experiments become impossible to interpret.

1. Direct prediction

The simplest execution policy is direct prediction:

import dspy

class ImproveSentence(dspy.Signature):
    """Improve one sentence while preserving meaning, entities, and voice."""

    sentence: str = dspy.InputField()
    editorial_goal: str = dspy.InputField()
    local_context: str = dspy.InputField()

    rewritten_text: str = dspy.OutputField()
    rationale: str = dspy.OutputField()
    confidence: float = dspy.OutputField(desc="Confidence from 0.0 to 1.0")

class DirectSentenceImprover(dspy.Module):
    def __init__(self) -> None:
        super().__init__()
        self.rewrite = dspy.Predict(ImproveSentence)

    def forward(self, sentence: str, editorial_goal: str, local_context: str = ""):
        return self.rewrite(
            sentence=sentence,
            editorial_goal=editorial_goal,
            local_context=local_context,
        )

Predict executes the declared signature without first extending it with an additional reasoning field. DSPy’s adapter still turns the signature into an LM interaction; “direct” describes the module strategy, not a claim about what computation occurs inside the underlying model.

For many tasks, this is the correct first implementation. It has fewer program-level moving parts than an explicit reasoning strategy and gives us a clean baseline to compare against.

2. Chain of thought under the same contract

Some failures may improve when the program gives the model an explicit reasoning field before the final outputs. We should not infer that cause from one bad answer; we should treat a different execution strategy as a hypothesis to test.

For those cases, we can try ChainOfThought:

class ReasonedSentenceImprover(dspy.Module):
    def __init__(self) -> None:
        super().__init__()
        self.rewrite = dspy.ChainOfThought(ImproveSentence)

    def forward(self, sentence: str, editorial_goal: str, local_context: str = ""):
        return self.rewrite(
            sentence=sentence,
            editorial_goal=editorial_goal,
            local_context=local_context,
        )

Current DSPy documentation describes ChainOfThought as a module that extends the supplied signature with an additional reasoning output field and then uses an underlying predictor to produce that reasoning plus the task outputs.

The declared task contract remains:

def run_case(program, case):
    return program(
        sentence=case["sentence"],
        editorial_goal=case["editorial_goal"],
        local_context=case.get("local_context", ""),
    )

direct = DirectSentenceImprover()
reasoned = ReasonedSentenceImprover()

direct_result = run_case(direct, case)
reasoned_result = run_case(reasoned, case)

The caller can invoke both modules with the same task inputs and rely on the same declared task outputs. But the returned Prediction objects are not literally identical in shape: ChainOfThought also exposes its added reasoning field. Application code should therefore depend on the declared outputs it needs, not on the assumption that every module returns exactly the same set of auxiliary fields.

3. Reasoning fields are not evidence

An explicit reasoning field can change task performance. It can also produce convincing reasoning attached to a bad answer.

For our editorial task, the added field might say:

This is why Writer’s contextual preference judge does not let the LM’s confidence decide acceptance by itself. It applies deterministic hard policies after the DSPy prediction. It records failure reasons and status. It compares DSPy decisions to human decisions during evaluation. The LM can inform the decision; it does not become the whole decision system.

The same caution will matter later when we evaluate optimized programs. A richer execution policy is a hypothesis, not a result.

4. What changes and what does not

The last row matters. If we test Predict on easy examples and ChainOfThought on hard examples, we learn little. If we change the signature, model, or metric at the same time as the module, attribution becomes weaker.

A controlled comparison should look like this:

5. Practical debugging

When a direct predictor fails, inspect the failure category before reaching for reasoning.

Reasoning is not a universal upgrade. It is an intervention with its own cost and failure modes. Use it when you have a testable reason to believe explicit intermediate reasoning may help, then measure whether it actually does.

6. A near-runnable comparison harness

The following code assumes DSPy and an LM are configured. It does not claim which module is better. It only creates a fairer comparison shape.

def compare_modules(cases: list[dict]) -> list[dict]:
    programs = {
        "predict": DirectSentenceImprover(),
        "chain_of_thought": ReasonedSentenceImprover(),
    }
    rows = []
    for case in cases:
        for name, program in programs.items():
            pred = program(
                sentence=case["sentence"],
                editorial_goal=case["editorial_goal"],
                local_context=case.get("local_context", ""),
            )
            rows.append(
                {
                    "case_id": case["case_id"],
                    "module": name,
                    "rewritten_text": pred.rewritten_text,
                    "rationale": pred.rationale,
                    "confidence": pred.confidence,
                    "reasoning": getattr(pred, "reasoning", None),
                }
            )
    return rows

Later we will replace manual inspection with metrics. For now the harness gives us two useful properties: both strategies receive the same task inputs, and we can see explicitly that ChainOfThought contributes an auxiliary reasoning field while preserving the outputs the application actually consumes.

What Usually Goes Wrong

Conclusion

We gained the ability to run the same task contract with different execution policies. That lets us compare mechanisms instead of comparing tangled prompt strings.

We removed the assumption that “better prompting” and “different reasoning strategy” are the same kind of change.

The remaining limitation is that real applications rarely consist of one LM call. A useful editor may need to analyze context, generate candidates, assess risk, and return a selected result.

That leads to composition.