Coding and Structured Generation
Can a student learn to generate functionally correct structured outputs from teacher demonstrations rather than merely copying syntax? Constrained sequence likelihood, schema validation, test-based verification, and a bounded E3 study where verification filtering recovers the teacher-demonstration gap.
Chapter 24 ranked documents. The output was an ordering, and the evaluation asked where the relevant document landed. This chapter changes the output object again. A program, a SQL query, or a JSON document is not liked or ranked; it runs, parses, or fails. Its correctness is a property of the artefact, checkable without asking any model for its opinion:
Can a student learn to generate functionally correct structured outputs from teacher demonstrations rather than merely copying the teacher’s syntax?
On our bounded study, the answer is yes with a qualification that matters. Students trained on teacher demonstrations underperform students trained on independently correct targets on every seed (0.367 against 0.391 mean functional correctness). Verification filtering — keeping only the teacher outputs that pass an independent check — recovers the gap on the mean (0.402), though it trails the oracle-target arm on one of the three seeds. Syntax is essentially perfect in all arms, so the difference is functional, not formal. The student that copies the teacher learns the teacher’s shape; the student that copies only the teacher’s checked outputs learns the task.
Code as conditional sequence modelling
A code or structured-output model is, mathematically, the same object as the instruction models of Chapters 17–22: a conditional distribution \(p_\theta(y \mid x)\) over output token sequences \(y\) given a prompt \(x\), factorised autoregressively:
This is why the chapter distinguishes three nested properties of an output. Syntax asks whether the tokens form a well-shaped artefact in the grammar (balanced braces, known tokens, terminated). Schema asks whether the shape carries the expected fields in the expected places (the right keys, digits where digits belong, no leading zeros). Functional correctness asks whether the artefact means the right thing (the sum is actually the sum). Each implies the previous one is satisfiable, but none implies the next. A model can master the first two and fail the third on every input — which is exactly what our students do on the inputs they get wrong.
Structured versus free-form outputs
Example A. The task this chapter measures. The prompt encodes three
integers, say \([3, 1, 2]\), as tokens
[BOS, INST_JSON, D3, D1, D2, EOS]. The correct completion is the
JSON document {"sum": 6}, encoded as
[LBRACE, K_SUM, COLON, D6, RBRACE, EOS]. The full row is these
sixteen tokens padded to length 32. The input space is all tuples of
2–5 values in 0–9: 111,100 possible prompts, of which each seed uses
800 for training, 200 for validation, and 400 held-out for testing,
pairwise disjoint by construction.
The output vocabulary has twenty tokens: the shared specials, the
JSON punctuation, two key tokens, and ten digit tokens. There are no
strings, no nesting, no comments, no imports. This is deliberately
the smallest task that still separates the three properties: the
template ({sum: __}) exercises schema, the digit sequence exercises
syntax, and the addition exercises function. A sum of 6 and a sum of
35 differ only in digit count; the grammar accepts both, and only
arithmetic tells them apart.
Contrast this with free-form generation, where the same prompt might accept “the sum is six”, “6”, “{"sum": 6}”, or a paragraph of working. Free-form evaluation must decide which of these count, and that decision is itself a judgement call. Structured evaluation outsources the decision to the grammar and the evaluator: the output either parses, validates, and computes correctly, or it does not. The chapter’s experiment uses the constrained form precisely so that no judgement call stands between the student’s output and its score.
Grammars, schemas, and the projection view
A grammar is a set of admissible token sequences. For our JSON-sum task, the admissible completions are exactly the strings matching
LBRACE K_SUM COLON digit+ RBRACE EOS
with the further restriction that the digit sequence has no leading zero unless it is the single digit zero. Call this set \(\mathcal{G}\). Unconstrained decoding samples from \(p_\theta(\cdot \mid x)\) over all sequences; grammar-constrained decoding projects onto \(\mathcal{G}\), typically by masking inadmissible tokens at each position and renormalising:
Schema validation is the executable form of the grammar. The
implementation checks, in order: terminal EOS present, braces
balanced, the K_SUM COLON header template, a non-empty digit run
with no leading zero, and no unknown tokens. Each failure maps to an
error category — syntax for unparseable outputs, schema for
well-formed tokens in the wrong template, semantic for a valid
document with the wrong sum.
A worked projection makes the masking concrete. Suppose at some
position the student’s raw distribution is 0.50 on D6, 0.30 on
RBRACE, 0.15 on K_SUM, and 0.05 spread over digit tokens, and the
grammar state admits only digits (we are inside the number field).
Masking zeroes the inadmissible mass (0.30 + 0.15 = 0.45) and
renormalises the rest: D6 becomes \(0.50/0.55 \approx 0.909\) and
the remaining digits share \(0.05/0.55 \approx 0.091\).
Figure 1. The worked masking step drawn as bars: 0.45 of raw mass
removed, D6 renormalised to 0.909. Worked arithmetic, not a measurement. The projection
did not make the student smarter; it made the inadmissible outputs
impossible. Any measured gain from constrained decoding is therefore
a property of the mask, not evidence the student learned arithmetic —
which is why our students are evaluated without it. The taxonomy is not decoration: it tells
the reader where the student’s capacity runs out. Here, the answer is
unambiguous. Across 3,600 held-out predictions (9 students × 400
tests), invalid-output rates sit at 0.003; schema validity rounds to
1.00. Every failure that matters is semantic: a valid {"sum": N}
with the wrong \(N\).
Why exact match is a weak proxy
Exact token-match accuracy asks whether the student’s completion is token-identical to the oracle’s. Functional correctness asks whether it parses, validates, and computes the right sum. On this task the two coincide numerically — canonical formatting (no leading zeros, fixed key order) means there is exactly one correct token sequence per input, so every functionally correct output is exact and vice versa. The manuscript reports both anyway, because their coincidence is a property of this grammar, not a general identity.
In general, exact match understates functional correctness whenever
multiple outputs are correct. A Python function that sums a list can
use a loop, sum(), or recursion; {"sum": 6} and {"sum": 6 }
differ by whitespace but compute identically (our grammar forbids the
second, but real JSON permits it). A student that finds a different
correct program scores zero on exact match and full marks on tests.
Conversely, exact match overstates correctness whenever the test set
leaks into training: a student that memorised the test rows scores
perfectly without computing anything. Both directions motivate the
same discipline: report exact match as a diagnostic, functional
correctness as the primary metric, and the train/test overlap audit
as the precondition for believing either.
Test-based verification generalises the point. HumanEval (Chen et al., P-057) evaluates code models by running generated programs against unit tests rather than comparing tokens — the preprint that introduced Codex and its HumanEval set frames functional correctness, not resemblance, as the measure of code generation. The chapter uses that design as background only: HumanEval is a benchmark and a model report, not a distillation study, and the register records its reuse and contamination risks. Nothing about Codex’s scores is cited here; no number from that paper appears as evidence. What transfers is the evaluation idea — an independent check the teacher does not grade — which our integer evaluator implements in miniature.
Teacher demonstrations and behaviour cloning
A teacher demonstration is a prompt paired with the teacher’s greedy output for that prompt, right or wrong. Training on demonstrations is behaviour cloning: maximum likelihood on the teacher’s behaviour, \(L = -\log p_\theta(\hat y \mid x)\) where \(\hat y\) is what the teacher produced. There is no reward, no ranking, no filter. The student learns the teacher’s distribution including its errors.
Three properties of demonstration learning matter before any experiment. First, the student inherits the teacher’s error rate as a ceiling on its training signal: if 3% of demonstrations are wrong, the best the student can do by pure imitation is bounded by that noise. Second, the student’s errors compound differently from the teacher’s: the teacher’s mistakes are scattered across its own competence frontier, while the student’s mistakes concentrate where its smaller capacity gives out — here, multi-digit sums. Third, the teacher’s agreement with the student measures imitation fidelity, not task success. A student that reproduces the teacher’s wrong sum scores one for agreement and zero for correctness.
Verification filtering breaks the first property. Instead of training on all demonstrations, train only on the ones that pass the independent check — here, the teacher outputs whose JSON parses and whose sum matches the oracle. The filter needs no model, only the integer evaluator both teacher and student are measured against. It is the direct descendant of Chapter 19’s verifier-guided selection, applied to structured outputs rather than reasoning traces. The cost is data: filtering discards the teacher’s wrong outputs (2–3% here), shrinking the pool. The question is whether cleaner data beats more data at matched update budgets.
flowchart LR
T[teacher demonstrations] --> V{verify: parse + oracle check}
V -->|pass| F[train on filtered set]
V -->|fail| X[discard with reason]
T --> B[train on all: behaviour cloning]
Figure 2. Arm B imitates everything; arm C trains only on what the independent check passes. Same teacher, same budget — the filter is the only difference.
The behaviour-cloning objective behind all three arms is worth writing down, because its gradient explains why the 3% matters. Given a training distribution \(q(x, \hat y)\) over prompts and demonstration outputs, cloning minimises the negative log-likelihood
On seed 50, \(\epsilon\) is about 0.03 (23 of 800 teacher rows wrong), yet B trails A by 0.050, more than \(\epsilon\) alone would suggest; amplification is one hypothesis. Filtering to \(\epsilon = 0\) raises C above B on every seed, though on seed 50 C still trails A (it trains on fewer unique rows, 777 against 800).
Example B — a small code task with unit tests
The JSON experiment is the measured instance; the code case is the
worked illustration the contract requires, presented without training
claims. Consider the specification: “write total(xs), returning the
sum of a list of integers.” The tests:
assert total([]) == 0
assert total([3, 1, 2]) == 6
assert total([-1, 1]) == 0
assert total([10, 20, 30]) == 60
Four candidate student outputs separate the properties. Candidate 1,
def total(xs): return sum(xs), passes. Candidate 2,
def total(xs) return sum(xs) (missing colon), is a syntax error —
it does not parse and no test runs. Candidate 3,
def total(xs): return len(xs), parses cleanly and returns 3 for
[3, 1, 2]; it is syntactically valid, type-correct, and wrong on
every non-trivial test. Candidate 4, def total(xs): return 6,
passes the second test and fails the rest — the memorised answer,
functionally correct on one input and evidence of nothing.
Candidate 3 is the chapter’s central warning in miniature: the most
dangerous student output is the one that looks most competent. It
runs, returns an integer, and never raises. Only the tests catch it.
Replace “unit tests” with “the integer evaluator” and “Python
function” with “JSON document” and this is exactly the E3 study
below, where every consequential error is a valid {"sum": N} with
the wrong \(N\).
Example C — the leakage case
Suppose the test prompt [3, 1, 2] also appears in training. A
student that memorised its training rows outputs {"sum": 6}
correctly without adding anything. Exact match is 1, functional
correctness is 1, and the measurement means nothing. The defence is
the overlap audit: before any training, the runner checks that the
train, validation, and test input-tuple sets are pairwise disjoint
and refuses to train on any overlap. All three seeds pass with zero
overlapping tuples out of 800/200/400. The audit is cheap, automatic,
and non-negotiable — the same discipline Chapters 12 and 13 apply to
teaching tasks generally. A structured benchmark without it is a
memorisation test wearing an evaluation costume.
Distribution shift deserves one paragraph of scope discipline. Our held-out split is an input split: test tuples are unseen combinations of seen values and seen lengths, not new values, new lengths, or new templates. A student that memorised single-digit sums would still fail two-digit ones; our test set contains both, but both come from the same generator. Generalisation to longer inputs, larger values, or a changed schema is not measured and not claimed.
The frozen E3 comparison
The corpus is self-created: all 111,100 tuples of 2–5 values in 0–9, from which each dataset seed draws disjoint 800/200/400 splits. The teacher is a width-32, 2-layer transformer (27,776 parameters) trained 1,500 optimiser updates on oracle rows; its held-out functional correctness is 0.890, 0.910, and 0.915, clearing the 0.50 calibration gate on all seeds. The students are width-16, 1-layer transformers (4,464 parameters, a 6.2-to-1 parameter ratio), one per arm per seed, each trained EXACTLY 600 updates with identical initialisations per seed.
Arms differ only in training rows: A on oracle completions, B on the teacher’s greedy completions, C on the functionally-correct subset of the teacher’s completions (777, 792, and 783 rows — filter rates 0.971, 0.990, 0.979). Validation selects each arm’s checkpoint by its own hard cross-entropy after the full budget; the test set is evaluation-only.
Per-seed held-out results (exact token match equals functional correctness under canonical formatting; both are reported):
| Seed | Arm | Exact / Functional | Syntax | Schema | Agreement | Invalid |
|---|---|---|---|---|---|---|
| 50 | A-oracle | 0.393 | 0.998 | 0.998 | 0.430 | 0.003 |
| 50 | B-teacher | 0.343 | 0.998 | 0.998 | 0.380 | 0.003 |
| 50 | C-filtered | 0.352 | 0.998 | 0.998 | 0.380 | 0.003 |
| 51 | A-oracle | 0.360 | 1.000 | 1.000 | 0.368 | 0.000 |
| 51 | B-teacher | 0.350 | 1.000 | 1.000 | 0.352 | 0.000 |
| 51 | C-filtered | 0.388 | 1.000 | 1.000 | 0.405 | 0.000 |
| 52 | A-oracle | 0.420 | 0.993 | 0.993 | 0.435 | 0.007 |
| 52 | B-teacher | 0.408 | 0.995 | 0.995 | 0.427 | 0.005 |
| 52 | C-filtered | 0.465 | 0.995 | 0.995 | 0.470 | 0.005 |
Aggregates (mean ± SD, descriptive, not significance tests): A 0.391 ± 0.030, B 0.367 ± 0.036, C 0.402 ± 0.058 functional correctness; agreement 0.411 / 0.387 / 0.418; invalid rates 0.003 throughout.
Figure 3. Mean functional correctness per arm: teacher demonstrations trail oracle labels on all seeds; the verified-filtered set recovers the gap. Frozen E3 measurements, descriptive means over 3 seeds. Token-level accuracy sits near 0.91 with NLL near 0.24 in all arms — the models predict most tokens well and fail at the arithmetic digits, which greedy decoding then gets wrong wholesale.
Three readings, each tied to a hypothesis. H1 holds on all seeds: oracle targets beat teacher demonstrations (+0.050, +0.010, +0.013). Copying the teacher costs correctness even when the teacher is strong (0.89+ validation) and its demonstrations are 97%+ correct; one reading is that the student amplifies the residual noise, though the experiment does not isolate that mechanism. H2 holds on all seeds: filtering beats raw demonstrations (+0.010, +0.038, +0.058) and lands at the oracle level (mean +0.011, mixed per-seed). Here, cleaner data beat more data at matched updates. H3 holds weakly: agreement exceeds functional correctness by about 0.02 in every arm. The overstatement is real but small — a reminder that agreement is the wrong metric, not a scandal.
Why agreement is not correctness, here concretely
For arm B on seed 50, teacher–student agreement is 0.380 and functional correctness is 0.343. The 0.037 gap decomposes into two populations: inputs where student and teacher agree and are both right (the bulk), and inputs where they agree and are both wrong (the teacher’s error reproduced). The second population is the interesting one: 3% of B’s training signal was wrong, and the student learned some of it faithfully. Agreement counts those faithfully wrong answers as successes. Functional correctness does not.
The filter’s effect is visible in the same decomposition for arm C. Removing the teacher’s wrong outputs removes the both-wrong population from training, and C’s agreement (0.380 on seed 50) now tracks its correctness (0.352) with the same small gap but a higher base. The lesson generalises beyond JSON: whenever the teacher is fallible, agreement rewards reproducing the failures. Only an independent check — tests, execution, arithmetic — separates the student that learned the task from the student that learned the teacher.
Cost, sandbox, and what was not run
Teacher training costs ~79–80 seconds per seed (1,500 updates); each student arm costs ~15.5 seconds per seed (600 updates); the study total is 385.5 seconds wall time with a 520.9 MB peak process high-water mark. Target-token exposure is ~134k per arm per seed. These are operation counts and wall-clock observations on one host, not prices; no API was used.
On sandboxes: the study executes nothing the model generates. The
“execution” is integer addition and grammar checks in repository code
over a closed token vocabulary — there is no eval, no exec, no
subprocess, no generated Python. The sandbox question the contract
raises is therefore answered by avoidance rather than by engineering:
a constrained task representation removes the arbitrary-code hazard
instead of containing it. A chapter that distils open-ended Python
generation would need the enforced sandbox the contract describes;
this one documents why its task does not, and states plainly that the
safety argument does not transfer to executable code generation.
What was not run: no multi-digit multiplication (abandoned pre-run as unlearnable at these budgets — recorded in the protocol, not a result), no out-of-distribution values or lengths, no pass@k sampling (the evaluation is single greedy decode per input), no representation-similarity analysis, no external replication, no independent review.
Rights, licences, and contamination
The study uses RM-117, VERIFIED_PERMITTED for
train_structured_distillation_e3. The corpus, teacher, and students
are self-created; no external code, model, API, or dataset is used.
A retrieval corpus is not the risk here — code is. Computer programs
are protected as literary works under Directive 2009/24/EC, which
permits observation, study, and testing but does not license training
by default (L-027). Training on licence-encumbered code, including
copyleft-licensed sources, can attach conditions to the resulting
model; generated code can reproduce training code verbatim.
None of that occurred here — every token the students saw was generated from the integer grammar — but the record exists because the next structured task may not be synthetic. The HumanEval discussion above carries the same caveat: a widely reused benchmark can be contaminated, and contamination turns functional evaluation back into memorisation testing. The legal findings are dated and action-specific; they are not legal advice.
Limitations
One synthetic JSON-sum task, three seeds, tiny transformers. Sum-only outputs capped at 45; multiplication, multi-field schemas, nesting, and open-ended code are unmeasured. Single teacher budget, single student budget, single filter rule. The held-out split is an input split, not a distribution shift. Token accuracy near 0.91 coexists with functional correctness near 0.39 — the models are fluent and frequently wrong, and greedy decoding converts token uncertainty into total failure. No external replication and no independent review.
Preparing for tool use
Generating a correct output is a single-step decision: prompt in, artefact out, check against the specification. Chapter 26 removes that comfort. An agent must choose tools, read their observations, recover from mistakes, and produce a final answer after several decisions — each one a chance to compound the last error. The JSON evaluator becomes a tool environment; the greedy decode becomes a policy rollout; the error taxonomy becomes compounding-error analysis. The progression the batch promised — generating correct outputs, then choosing correct actions — starts with the machinery this chapter built: a deterministic environment, an independent success check, and the discipline of never grading the student against its teacher.
References
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. O., Kaplan, J. et al. (2021). Evaluating Large Language Models Trained on Code. arXiv preprint. arXiv:2107.03374. https://arxiv.org/abs/2107.03374
Legal sources
Directive 2009/24/EC of the European Parliament and of the Council of 23 April 2009 on the legal protection of computer programs. In force 5 May 2009. https://eur-lex.europa.eu/eli/dir/2009/24/oj (retrieved 12 November 2026). Cited as L-027 for the observation/study/testing context; not legal advice.