Order Changes Meaning

A harness team ships a minor upgrade. Nothing is added to the context and nothing is removed. The release notes mention only a rendering change: tool results now appear above the project rules instead of below them, so that fresh observations sit closer to the current request. Within a week, the team’s deployment-safety rule, which the agent obeyed for months, starts getting violated. Nobody changed the rule. Nobody changed the model. The information in every bundle is byte-identical to last week’s. Only the arrangement changed, and the behaviour changed with it.

Chapter 5 established that position affects usability. This chapter asks what, exactly, ordering controls, and the answer is larger than position. A context bundle is not a set of information from which the model draws what it needs. It is a sequence, and the sequence encodes several distinct things at once: where items fall, which instructions govern, what sits next to what, how events are grouped in time, and which spans stay stable across turns. Teams that assemble bundles by appending whatever arrived most recently are encoding all of those things accidentally.

Six things the word order means

The chapter separates six meanings before prescribing anything, because each has different evidence and different remedies.

Position is where an item falls in the sequence: early, middle, or late. The detailed evidence belongs to Chapter 5 and is not repeated here: the U-shaped middle trough of Lost in the Middle on QA and retrieval tasks, the RULER finding that claimed length outruns demonstrated length, the NoLiMa collapse without lexical shortcuts. Chapter 6 inherits the conclusion and reuses the instrument: position is the manipulated variable of the Family B fixture, with evidence moved through normalised positions while content is held constant. What this chapter adds is the observation that position never travels alone. Moving an item later also moves it nearer the request, farther from the instructions, and into a different cache span. A position experiment that does not record those confounds cannot say which aspect of “later” did the work, which is why the proposed fixtures log adjacency and stability alongside position.

Precedence is which instruction governs when instructions disagree. Here the book has first-party documentation rather than benchmark inference. OpenAI’s Model Spec, first published in May 2024 and updated in February 2025, specifies an instruction hierarchy, described as a chain of command: system instructions outrank developer instructions, which outrank user instructions, with the developer message taking precedence on direct conflict. The worked example is prompt-injection-shaped, a user telling the model to ignore previous instructions while the developer message forbids it, and the specified verdict favours the developer. Two qualifications matter. First, this is OpenAI’s documented policy for its models, not a universal law of language models; providers differ, and precedence in practice depends on training as much as on documentation. The book will not invent a cross-vendor hierarchy. Second, precedence is a property of roles and training, not of sequence position. A harness that relies on placing its most important rule last, hoping recency will enforce what authority should, has confused the next two meanings with this one.

Adjacency is what sits next to what. A claim followed immediately by its evidence is a different object from the same claim separated from its evidence by three thousand tokens of unrelated history, even though the set of tokens is nearly identical. The mechanism is straightforward: intervening material competes for the retrieval that connects the two, exactly the interference Chapter 5 measured at bundle scale. The natural unit is the pair that must be used together: tool call with its result, rule with the operation it constrains, example with the query it demonstrates, assertion with its citation. Assembly that scatters pairs across the bundle pays the interference tax on every one of them. The proposed adjacency fixture below tests exactly this, claim with evidence adjacent versus separated, holding everything else fixed.

Grouping is whether related information is clustered or scattered. Ten project rules in one block present one object for the model to consult; the same ten rules dispersed through unrelated context present ten retrieval problems. Chroma’s structure-sensitivity finding is the adjacent evidence worth recording honestly: models performed consistently better on shuffled haystacks than on coherent ones, an effect the authors leave unexplained. That result cuts against any glib claim that clustering always helps, and the chapter preserves the disagreement rather than smoothing it. The plausible reconciliation, stated as hypothesis rather than fact, is that coherence in irrelevant material creates competing structure that distracts, while coherence in task material creates usable structure that guides. If that distinction holds, the rule is not “cluster everything” but “cluster what must be used together, and keep the rest from forming rival structure.” The grouping fixture tests the first half; the second half awaits better evidence.

Temporal order is how events across turns are sequenced: chronological, reverse-chronological, or grouped by semantic role. Most harnesses default to chronological history because conversation happened that way, but chronological is a choice with consequences. Early turns contain the task framing the model may need most; late turns contain the freshest observations competing most directly with the request. Reverse-chronological layouts privilege recency at the cost of narrative. The few-shot literature gives the oldest relevant warning: Zhao et al. showed in 2021 that example order alone could swing GPT-3 accuracy from near chance to near state of the art, with a measurable bias toward answers placed near the end of the prompt. The population is GPT-3-era classification, not current agentic bundles, and the chapter claims no more than the mechanism: sequence position biases selection among alternatives, and alternatives-among-history is what a long trace presents at every turn. Which temporal layout serves coding-agent repair is an open question this chapter leaves to a future fixture family, as recorded below, not a settled rule it pretends to know.

Stable versus dynamic placement is which spans change across turns. This meaning belongs partly to economics rather than behaviour, and the chapter treats it as a constraint on the other five. Prefix-matching caches reuse computation over matching leading spans, so material that never changes earns its keep early, where it stabilises the prefix, while material that changes every turn belongs late, where its churn invalidates the least. OpenAI’s caching documentation makes the prefix-match requirement explicit, and Chapter 9 will build the full mechanics on it. Here the point is narrower: ordering decisions made for behavioural reasons, instructions first, request last, have economic side effects, and ordering decisions made for economic reasons have behavioural side effects. A layout must satisfy both, which is one more reason the layout should be deliberate rather than emergent.

One move, six effects

The six meanings are distinguished analytically but never occur separately, and a worked example shows why experiments must log all of them. Take a bundle whose tail currently reads: project rules block, then two file excerpts, then three tool results, then the current request. The harness team moves the project rules to just above the request, reasoning that the rules govern the imminent action and should be fresh in the model’s attention.

That single move changes all six things. Position: the rules travel from early-middle to late. Precedence: unchanged on paper, since roles and text are identical, but the team is now relying on recency to do precedence’s work. Adjacency: the rules gain adjacency to the request and lose it to the task state they used to follow; the file excerpts gain adjacency to each other. Grouping: the rules leave their block-mates if any standing instructions stayed behind, splitting one governed object into two. Temporal order: untouched, since rules carry no timestamp. Stability: if rules were part of the stable prefix, the prefix just shortened, and every cached span after the old rules position invalidates.

If behaviour then changes, which meaning caused it? Without the ordering trace, the team will credit the meaning they intended, recency helping governance, and miss the others. The fixture discipline of this chapter exists to prevent exactly that misattribution: manipulate one meaning, freeze or counterbalance the rest, log all six regardless. Most harness changes in the wild manipulate all six at once and learn nothing.

Temporal order is a choice with consequences

Most harnesses present history chronologically because conversation happened that way, but the layout that records events faithfully is not necessarily the layout that best serves the next computation. Three arrangements compete. Chronological order preserves narrative: the task framing of early turns stays where the story put it, at the cost of burying that framing under everything since. Reverse-chronological order privileges the freshest observations, matching the recency bias the few-shot literature measured, at the cost of detaching conclusions from the reasoning that produced them. Role-grouped order clusters by function, all tool results together, all decisions together, regardless of when each occurred, at the cost of breaking the causal chain that explains why each item exists.

No arrangement dominates because each serves a different use of history. Narrative serves diagnosis: what did we try, in what order, and what did each attempt teach? Recency serves action: what is the latest state of the world I must respond to? Grouping serves consultation: where is every rule, every decision, every result of one kind? A repair trajectory mid-flight mostly needs the second and first; a compliance audit of the same session needs the third. The harness default of chronological-everything is therefore not neutral ground but an unexamined win for narrative over the other two, and teams surprised that models overweight stale early framing or underweight buried decisions are meeting their default’s consequences.

The fixtures in this chapter hold temporal order fixed while testing other meanings, which is the correct discipline for one-variable interventions and leaves temporal order itself largely untested. The book records the gap openly: which temporal layout serves coding-agent repair, and whether the answer changes with trajectory length, is a hypothesis awaiting its own fixture family, not a recommendation smuggled inside the canonical layout below.

Salience is not authority

The six meanings collapse into one confusion so reliably that it needs its own section. Teams observe that later items seem to weigh more, or that earlier items seem to set the frame, and conclude that position confers importance. Then they use position as a control knob for authority: the rule they most want obeyed goes last, the context they most want ignored goes first, and the bundle becomes a system of hints about standing expressed through sequence.

The evidence says position confers salience, not authority, and the two diverge exactly where it matters. Wang et al. demonstrated in 2023 that LLM judges’ verdicts on identical candidate responses could be flipped by reordering the candidates: Vicuna-13B beat ChatGPT on 66 of 80 queries under order manipulation, with ChatGPT itself as evaluator. Nothing about the candidates changed. The order changed what the judge noticed, emphasised, and preferred. That is salience: allocation of processing, not assignment of standing. A later, louder tool result does not outrank an earlier project rule; it merely shouts over it. When the shouting works, the team concludes the rule was unclear and rewrites it, when the defect was architectural: authority was encoded nowhere except in the team’s hopes about sequence.

Hence three sentences the chapter states as design rules, each following from the evidence rather than from taste:

recent does not mean authoritative
early does not mean important
loud does not mean governing

A future context compiler will need explicit metadata for what position currently smuggles: authority fields recording whose instruction an item is, scope fields recording what it governs, precedence resolved by policy rather than by slot. Chapter 2’s instrument already banks those fields; this chapter explains why they cannot remain write-only forever. Full adjudication machinery belongs to Chapter 19, which is foreshadowed here and not taught. The present claim is only that position is a lossy, ambiguous channel for authority, and that every bundle which uses it as such contains a latent instruction conflict waiting for the right distractor.

No universal placement rules

The chapter now earns its restraint. The literature supports patterns, not placements, and the patterns come with populations attached. Compressed to one view:

In particular the chapter refuses three sentences outright. “Always place important information first” confuses authority with primacy and fails against recency evidence. “Always place the request last” happens to match common harness practice but has not earned its status as law; it is a candidate canonical choice to be tested, not a finding. “Middle context is ignored” overstates a degradation curve into a deletion claim the data never supported. Where benchmarks disagree, the Chroma coherence result against clustering intuition, the chapter keeps both results visible. A compiler built on smoothed-over evidence inherits the smoothing as brittleness.

Proposed experiments: one variable at a time

The designs below refine and extend Chapter 5’s Family B fixture rather than inventing a new experimental world. All fixtures share the Family B controls: fixed model and version, frozen instruction bundle, fixed tools, deterministic bundle constructor, pre-registered trial counts, and the separated measurements of task success, evidence recovery, instruction adherence, unsupported claims, and latency. The recovery-versus-reasoning split carries over unchanged: a position failure that leaves evidence unrecovered is a different mechanism from one where recovered evidence is misused, and the fixtures must keep them distinct.

Fixture 1: normalised position. The frozen bundle contains fixed instructions, a fixed task, fixed critical evidence, fixed distractor material, and a fixed total token count where practical. The critical evidence moves through five normalised positions, 5, 25, 50, 75, and 95 per cent of rendered length, with token deltas reported per position so length confounds are visible. This is Family B made precise, and it tests the book’s recency-crowding hypothesis from Chapter 5: late-position crowding in agent-shaped bundles rather than the classic middle trough.

Fixture 2: evidence adjacency. Two conditions only. In the adjacent condition a claim appears immediately followed by its supporting evidence; in the separated condition the same claim and the same evidence are divided by a fixed span of unrelated material, with total tokens matched. Instructions, authority markings, and positions of all other items are held fixed. The comparison isolates adjacency from position as cleanly as construction allows.

Fixture 3: grouped versus dispersed rules. The same set of project rules appears either as one contiguous block or dispersed at fixed intervals through unrelated context, token counts matched, rule wording byte-identical. This tests grouping, not content: any behaviour difference is attributable to clustering alone.

Fixture 4: instruction and evidence order. Move task data earlier or later while holding instruction text, roles, and authority markings fixed. This tests whether data placement affects behaviour independently of instruction precedence, the converse of the salience-authority confusion: if moving data without touching instructions changes adherence, the mechanism is salience, and any harness relying on instruction position for enforcement is standing on the wrong variable.

No fixture combines manipulations. In particular, distractor placement must be counterbalanced or frozen across position conditions, per Chapter 5’s factorial warning: a distractor that sits near the evidence in one condition and far in another turns a position experiment into an interference experiment wearing a position costume.

What each fixture measures, and why

The shared measurement set needs per-fixture assignment, because each fixture fails in a characteristic way and the wrong metric misses it. Fixture 1, normalised position, is decided by critical-evidence recovery across positions: a U-shaped or late-crowded recovery curve is a position signature, while flat recovery with varying task success points past position toward reasoning or interference. Fixture 2, adjacency, is decided jointly by recovery and unsupported claims: separated evidence should show either failed recovery or, more tellingly, successful recovery of the claim with confabulated support, the model answering from the claim’s gist plus neighbouring filler. Fixture 3, grouping, is decided by instruction adherence on the dispersed rules: which rules drop first when scattered reveals whether dispersion costs uniform attention or severs specific rule-to-task links. Fixture 4, data order with fixed instructions, is decided by adherence with recovery held constant: if the evidence is recovered in all conditions but adherence varies, the mechanism is salience acting on identical authority, the chapter’s central distinction made measurable.

Latency enters as a secondary signal rather than a verdict. Slower responses on separated or dispersed conditions suggest costlier retrieval even when the answer survives, which matters for the economic budget of Chapter 4 even where behaviour holds. A layout that preserves accuracy at double the latency has not preserved the bundle’s value; it has converted a behavioural cost into an economic one, and the measurements should record the exchange rate.

Toward a canonical layout

Experiments test; production must still ship. Near the chapter’s end, the book introduces the idea that production systems benefit from a canonical layout: not a universal ordering, but a repeatable one, versioned with the harness and constant across turns except where the fixtures above justify variation. An illustrative candidate, and only that:

stable instructions
project constraints
tool definitions
task state
selected evidence
recent observations
current request

Every line of that sketch is a hypothesis. Stable material leads because prefix stability rewards it economically and because authority conventionally precedes data. Task state precedes evidence because framing scopes retrieval. Recent observations sit late because they are the most volatile and the most immediately relevant. The current request closes because the response continues from it. Each “because” is either established background from earlier chapters or a claim the fixtures are designed to check, and any line that fails its fixture gets moved or removed.

The sketch also predicts specific failure modes for its own violation, which is what makes it testable rather than decorative. Instructions buried after volatile observations should show salience interference: later noise shouting over earlier governance. Evidence scattered away from its claims should show recovery drops at matched token counts. Task state placed after the evidence it frames should show mis-scoped retrieval, the right mechanism applied to the wrong understanding of the job. Recent observations pinned early for stability should show staleness persistence, old readings treated as current. Each prediction names the fixture that checks it and the measurement that decides. A canonical layout that cannot state what its violation looks like is a preference; one that can is an instrument.

The important idea is not the sketch but the property: context order deliberate and reproducible rather than accidental, so that behaviour changes can be attributed to content changes rather than to silent reordering.

That property gives Context Lab one more future measurement, recorded here as a requirement rather than an implementation: a position and ordering trace, capturing each item’s normalised position, its group membership, and its adjacency relations per invocation, alongside the existing eleven fields. Without the trace, a future ordering regression is undetectable: the bundle contents match, the behaviour differs, and nothing in the records explains why.

The question ordering cannot answer

Suppose the layout is settled and every item sits where evidence says it belongs. Capacity pressure arrives anyway, as Chapter 4 guarantees it will, and something must give. Should the safety rule and the compiler log yield in the same way? Should the architecture decision and the abandoned hypothesis be summarised by the same policy? Ordering tells us where information appears. It says nothing about what kind of information it is, and pressure treats all kinds alike unless something intervenes. That intervention is classification:

Even if we know where information should appear, should every piece of information be treated the same way when context pressure rises?

References

• Liu, N. F., Lin, K., Hewitt, J., et al. “Lost in the Middle: How Language Models Use Long Contexts.” TACL 2023; arXiv:2307.03172. Peer-reviewed. Position sensitivity; inherited result, not re-argued here. https://arxiv.org/abs/2307.03172
• Zhao, T. Z., Wallace, E., Feng, S., Klein, D., Singh, S. “Calibrate Before Use: Improving Few-Shot Performance of Language Models.” ICML 2021; arXiv:2102.09690. Peer-reviewed. Example-order swings; recency bias; calibration remedy. https://arxiv.org/abs/2102.09690
• Wang, P., Li, L., Chen, L., et al. “Large Language Models are not Fair Evaluators.” Preprint, arXiv:2305.17926, v2 August 2023. Judge verdicts flippable by candidate order; balanced-position calibration. https://arxiv.org/abs/2305.17926
• OpenAI. “Introducing the Model Spec.” May 2024, updated February 2025. First-party documentation. Instruction hierarchy: system over developer over user; provider-specific, not universal. https://openai.com/index/introducing-the-model-spec/
• OpenAI. “Prompt caching.” Official documentation, verified September 2026. Prefix-match requirement underlying stable-first placement. https://developers.openai.com/api/docs/guides/prompt-caching