Chapter 9 finished the first arc. Seven layers now stand between raw history and an answer: retrieval, a persistent graph, associative pathways, routing, support lineage, temporal validity, open-loop status. Each earned its place against the failure class it claimed to fix. None has been asked the question that decides whether any of it was worth building.

A context window is small and history is not. Every turn, something chooses which fraction of the past the model is allowed to see. So far that choice has been made by one mechanism: similarity to the words of the request. The enrichment of Chapters 4 to 9 sits behind that gate, and a gate that only understands wording cannot use it.

This chapter is the book’s sixth question — what should I remember right now? — and it is the first chapter whose experiment puts every earlier layer on trial at once.

The failure the baseline cannot see

The controlled corpus for this chapter is the book’s own programme: sixty-eight memory units across three projects, where memory-book sits beside two neighbours, writer and cocoder, that talk about memory, context, evidence, retrieval and evaluation in their own vocabulary. The overlap is deliberate. A leakage measurement over corpora that share no words would measure nothing.

Ask the Chapter 3 system a question:

Review Chapter 10 and tell me what we should do with it.

Now ask it twice, under two different pieces of work.

The first is a publication review: the chapter is in the release queue, and the job is prose, citations and front matter. The evidence that matters is the chapter’s own text, its concept notes, its references, the style conventions and the neighbouring handoffs.

The second is an architectural review: the chapter’s place in the book is in doubt, and the job is to decide whether it belongs at all. Now the evidence that matters is the rule that mechanisms earn their complexity experimentally, the Chapter 3 baseline they must beat, the Chapter 2 measurement contract, the Chapter 8 and Chapter 9 results, the six-question spine — and, from the chapter itself, only the sentence recording that it has no committed run.

Same corpus. Same query. Same reader. Same token budget. The two jobs need almost disjoint evidence.

Book result. At a fixed 1,500-token estimated budget, the Chapter 3 hybrid baseline returns the identical bundle under both objectives: twenty-one units, overlap 1.00. So do the recency-augmented and project-scoped variants. Query-only retrieval is not weakly sensitive to the current goal; on this fixture it is exactly insensitive, because the goal never reaches it.

That is not a ranking defect to be tuned away. The baseline is working correctly. It was asked which passages resemble a question, it answered, and the question was the same both times. The information that would separate the two cases — what is being done, and why — was never part of its input.

Similar findings are now appearing across the literature. Intent-driven selection reranks retrieved memory by task benefit rather than resemblance (MemGuide); context-dependent frameworks condition recall on the current task and environment (CDMem); and trustworthy-memory work argues the sharper version, that a semantically related memory can be actively inappropriate, making memory admission a control boundary rather than a ranking detail (MemGate). The premise is not this book’s invention. What the book adds is the measurement against its own baseline, and the insistence that every selection be explicable afterwards.

Two lenses, not one

The naive repair is to put the current goal into the retrieval query. That fails for a reason worth stating: a project has standing commitments that no single request mentions. This book compares against strong RAG whatever anyone asks today. A medical study has protocol constraints; a codebase has conventions. Folding those into each request either repeats them forever or loses them.

So the design separates configuration from observation.

A ProjectFrame is durable, versioned, and written by a person. It holds what generally matters in a project: its purpose, its standing objectives and constraints, which projects are in scope, and — the part that does the work — an ordered preference over evidence classes for each shape of work the project does. It is edited rarely and never contains the current request.

A WorkFrame is ephemeral and rebuilt every time. It holds the objective in force now, the kind of work it is, the constraints currently active, and the signals it was built from.

The signal distinction matters more than it first appears. A work frame is assembled from WorkSignal objects, and a signal may be a user message, a project-state observation, a tool result, a failing test, a scheduled job, an event, or another agent’s task. Nothing in the design requires a conversation. In the fixture suite the store-configuration frame is built from a failing test and the release-readiness frame from a scheduled job; neither has a prompt anywhere. That is why the object is not called a prompt frame. It is also not called a thought frame: the system has no access to a model’s reasoning, and a name that implies otherwise would be a claim the book cannot support.

Every field of a work frame cites the signal identifiers that licensed it. A field with no signal behind it is a guess, and the schema makes the guess visible rather than plausible.

ProjectFrame     durable project policy      (configuration)
      │
WorkSignal(s)    message / event / tool result / failing test / job
      │
WorkFrame        what is being done now      (observation)
      │
ContextBundle    the exact evidence the model sees
      │
ContextTrace     why every candidate was admitted or rejected

Importance is not a field

The archived draft of this chapter refused a priority field on an intention record, on the grounds that importance belongs to retrieval policy rather than to the stored object. That refusal is now load-bearing.

No memory unit in this layer carries a stored importance, and a unit test enforces it. Importance is computed against a frame at selection time and written to the trace. The same unit is required evidence under one objective and a distraction under another, and both facts are recorded without either being stamped on the memory.

A staged policy, not a single score

The obvious implementation is a weighted sum over relevance, recency, goal fit and the rest. Generative Agents established the separation of those components for moment-to-moment retrieval, and the separation is right. The collapse into one number is what this chapter avoids, for a reason specific to its thesis: a scalar cannot be blamed. When a bundle is wrong, a weighted score names nothing.

The pipeline is therefore staged, and each stage records what it did.

Candidate retrieval runs the Chapter 3 shape unchanged — BM25 and bge-m3 dense search fused by reciprocal rank fusion — at high recall rather than top-k. From the work-frame rung onward the frame also participates here: the objective runs as a second query and is fused with the first, and each evidence class the project frame prefers for this work type is guaranteed a minimum number of candidates. That addition was not in the first design. It was forced by the first run, which showed required evidence failing at the retrieval stage under the release-readiness objective: the open-loop records were never candidates at all, because “what is left to do?” shares almost no vocabulary with the text of an open loop. A frame that only reranks cannot repair what retrieval never surfaced.

Hard eligibility then removes what is not admissible at all: out-of-project units; units superseded as of the frame time whose successor is also available; and units of a class the work type does not list that also lack query support.

Signals are computed and stored separately — query relevance, goal relevance, project relevance, temporal validity, evidence strength, open-loop relevance, adjacency, recency, redundancy — and candidates are then bucketed by the class preference the project frame declares for this work type, ordered by retrieval score inside a bucket. Coalescing and budget admission finish the job.

The signal implementations reuse the earlier layers rather than reimplementing them. Temporal validity is Chapter 8’s supersession. Evidence strength is Chapter 7’s distinction between support, restatement and refutation. Open-loop relevance is a Chapter 9 expectation, weighted by whether the work type has any use for unfinished work — which is the thesis in miniature: an open loop is not automatically important, it is important when the work makes it so.

Coalescing may remove repetition, nothing else

One rule governs compaction, and it is a restriction rather than a capability.

Restatements fold. When four planning notes echo one methodological rule, Chapter 7’s ECHO_OF lineage lets them stop occupying four slots — but their source references join the surviving item, so the reader still sees where the restatements came from. Disagreement never folds: a refuting unit becomes its own item that names what it contradicts. Supersession becomes a note rather than a second passage, so a bundle can say SQLite, superseded by PostgreSQL without spending budget on the stale text and without pretending the earlier state never held.

The caution here is borrowed rather than discovered. Anthropic’s context-engineering guidance notes that aggressive compaction discards detail whose importance was not yet apparent, and recommends maximising recall before improving precision (Anthropic, 2025, vendor-authored). Every coalescing operation here is recorded in the trace and reversible from it.

The trace is the artifact

A bundle shows what the model saw. It cannot answer the question a failure actually raises.

Each run freezes a ContextTrace: the project frame and its version, the work frame, the policy version, the query, the standpoint time, the budget, the retriever, every candidate with its separated signals, the stage that decided it, a reason code, and the bundle digest. Reason codes are preferred to model-written explanations, because an explanation generated after the fact is not evidence about the decision.

The requirement that matters is easy to state and easy to get wrong: the trace must carry full decision evidence for rejected candidates, not only admitted ones. An early version of this layer computed signals after the eligibility stage, so anything dropped there recorded zeroes. The bundle still looked correct and the reason codes were still present, but the trace could answer what did the system see? and could no longer answer why did it not see the thing it needed? — which is the only question a failure actually asks. Every candidate now carries its retrieval source, its raw retrieval score, its eligibility decisions, its frame signals, the policy stage that disposed of it, the budget state at that moment, and its final disposition, whether it was admitted or not.

The trace then turns “the answer was wrong” into an attribution:

E0  required evidence absent from memory
E1  present, but frame-conditioned retrieval never surfaced it
E2  surfaced but ruled ineligible
E3  eligible but lost during ranking or to the budget
E4  superseded or harmful evidence admitted
E5  coalescing damaged a needed distinction
E6  good context supplied; the reader failed

This is the Chapter 2 requirement — failures separated by stage before a mechanism is credited or blamed — applied to the stage the instrument had not yet reached. Two of these stopped being hypothetical during this chapter’s runs: the release-readiness objective failed at E1 until retrieval became frame-aware, and the chapter’s own experiment-pending marker failed at E3 under architectural review. Neither is visible from a bundle.

The experiment

Eleven tasks across seven fixture families, each carrying a hidden ledger that labels corpus units MUST, SHOULD, OPTIONAL, DISTRACTOR or HARMFUL. Context quality is scored against that ledger before any answer is generated, because a strong reader can hide a bad bundle.

Reader, corpus, query and budget are fixed. Only context construction changes.

C0  strong RAG, query only                  (the Chapter 3 baseline)
C1  C0 + a recency preference over recent work
C2  C0 restricted to the project scope
C3  C2 + the ProjectFrame's default class policy
C4  C3 + the WorkFrame: objective-expanded recall, work-type policy
C5  C4 + the Chapter 7, 8 and 9 signals and their candidate pulls
C6  C5 + adjacency, coalescing and redundancy
CO  the ledger oracle: what the budget physically permits

CO is not a system. It reads the hidden ledger and admits required evidence first, so the ladder is read against an attainable ceiling rather than against 1.00.

Book result (frozen run ch10-20260920T163314Z-context-frames, 68 units, 11 tasks, 1,500-token estimated budget, bm25 + bge-m3 + rrf60, reader llama3.1:8b at temperature zero). Means over tasks:

Four things in that table are worth more than the headline.

Project scope is the cheapest win and it is not the interesting one. Restricting candidates to the project removes cross-project leakage entirely (0.212 to 0.000) and lifts precision by fourteen points without touching recall. On the three-project fixture where the same question — how should we improve the context system? — is asked of memory-book, writer and cocoder, mean leakage falls from 0.667 to 0.000. This is scoping, which production memory systems already do, and it earns its place on safety grounds rather than quality.

The project frame alone makes things worse. C3 is the ProjectFrame applied without knowing what work is being done, and its must-include recall falls to 0.621 — below the baseline it was meant to improve. A standing policy that cannot see the current objective applies the wrong standing policy. That is a negative result about the configuration half of the design, and it is the reason the two frames are separate objects rather than one.

The work frame is where the improvement is. C4 lifts must-include recall from 0.773 to 0.901 and precision from 0.194 to 0.460, at slightly lower token cost. Against the oracle ceiling, the best condition closes about a third of the precision gap the baseline leaves open — an improvement, and a reminder of how much of the budget is still spent on evidence the work does not need.

The earlier layers pay narrowly, not broadly. C5 adds the Chapter 7, 8 and 9 signals and moves mean precision by 0.014 while driving harmful admission from 0.026 to 0.000. C6 adds adjacency, coalescing and redundancy and moves mean precision down by 0.010. On these fixtures, at this budget, the accumulated machinery buys a specific safety property and almost nothing else.

Where the goal changes the evidence

The same-query fixture is the chapter’s signature, and its numbers are unambiguous.

The single unit both objectives keep is the record that Chapter 10 has no committed run — which is, correctly, evidence for both jobs.

Three narrower fixtures probe specific mechanisms.

Recency against purpose. Several sessions of intention-extraction discussion precede a stated change of direction: return to the baseline and ask what the next mechanism improves. Recency says the intention discussion is the most important thing in the corpus. C0 and C1 reach must-include recall 0.750 and spend budget on a session note from the abandoned direction; C4 reaches 1.000 and admits none of them. Nothing here is superseded — the sessions happened and their record is accurate. They are simply no longer useful, which is a judgement only the current objective can make.

Current against superseded. A superseded SQLite design is lexically excellent for a question about the baseline store. C0, C2 and C4 all admit it; C5 rejects it with the reason code SUPERSEDED_SUCCESSOR_AVAILABLE, while keeping the current PostgreSQL record — must-include recall stays at 1.000 and harmful admission falls to 0.000. Chapter 8 earns its rung here, and the successor pull is why: a policy that can only silence the stale unit, without a current one to put in its place, is choosing between a wrong answer and no answer.

Unfinished work, conditioned. The question what is left to do? is asked twice, once under a release sweep and once under a prose objective. C0 returns the identical bundle for both — overlap 1.000 — containing no open-loop records at all. C4 admits three open loops under the release objective and none under the prose objective, with must-include recall 0.667 and 1.000 respectively. Chapter 9’s material is not automatically important; it becomes important when the work makes it so.

Restatement. Four planning notes echo one methodological rule. C0 admits all four, redundancy 0.190. C5 admits none of them, the echo demotion having moved them below the budget line. C6 folds three into the rule’s own item while preserving their sources, and every bundle item in every task retains at least one source reference across the whole suite. Notably C5, which simply demotes echoes, is cleaner on this fixture than C6, which coalesces them — the folded members still count as admitted units in the metric even though they occupy no separate item. The coalescing machinery keeps provenance as promised; on this corpus it does not yet buy anything the demotion did not.

The hazard

A frame that is wrong is more dangerous than no frame, and the experiment measures how much.

Book result. With the work type deliberately misassigned, must-include recall on the architectural review falls from 0.833 to 0.333, against a query-only baseline of 0.667. On the publication review it falls from 0.750 to 0.000, against a baseline of 0.750. In both cases the framed system selects worse than the unframed one.

This is the structural risk the trustworthy-memory literature identifies, arriving here as a number rather than a warning: a memory layer that decides what the model may see is a control channel, and a control channel that is misconfigured does not degrade gracefully. It confidently supplies the wrong evidence. Two consequences follow for anything built on this design: the query-only condition has to remain available as a fallback, and the trace has to be inspectable, because the failure mode looks exactly like a confident success from the outside.

The frame builder is therefore the layer’s real failure surface, and the run measures it directly rather than assuming a correct frame. Three builders were compared on the same eleven tasks: frames a person declared, frames a deterministic cue-count classifier produced, and frames the reader model produced from the same signals under a constrained JSON schema.

Book result.

frame builder	work-type accuracy	mean must-include recall
declared by a person	—	0.901
cue-count classifier	0.727 (8/11)	0.871
model-inferred	0.636 (7/11)	0.780
no frame at all (C0)	—	0.773

With a model-inferred frame the entire advantage over query-only retrieval very nearly disappears: 0.780 against a baseline of 0.773.

That is the most important number in the chapter, and it is not a flattering one. Everything above depends on a frame being approximately right, and the realistic builder is the weakest link by a wide margin.

The per-task detail shows why a mean accuracy figure is the wrong summary. The model misread four of the eleven work types. Three of those errors cost nothing at all — two architectural reviews read as implementation review and a third read as experiment analysis still reach must-include recall 1.000, because the required evidence is reachable under either preference ordering. The fourth costs everything: a prose-finishing task read as implementation review scores 0.000. One catastrophic misreading, not a general degradation, accounts for most of the gap between 0.901 and 0.780. Frame errors are not evenly distributed in their consequences, and the rate conceals the distribution.

A second effect is easy to miss. On one task the model chose the correct work type and still lost half the required evidence, because it also wrote its own objective sentence, and the objective feeds candidate generation and the overlap term. Classifying the work correctly is necessary and not sufficient; how the objective is phrased moves selection too.

The asymmetry underneath all of this is what makes a frame different in kind from a query. A poor query produces mediocre retrieval, and the evidence is still in the candidate set to be recovered by reranking, by a follow-up, by a wider pool. A wrong frame decides which evidence is a candidate at all, and nothing downstream can recover what was never proposed. That is why the same magnitude of error is survivable in one place and not in the other.

Design consequence, not a measured result. The remedy these numbers point to is that a work frame needs an abstention path of its own. A frame would carry how it was established — declared, inferred with confidence, inferred ambiguously, or unknown — and conditioning would weaken as that degrades, falling back toward query-only retrieval when the signals do not determine the work. The measured harm argues for it; this chapter did not build or test it, and it is recorded as the layer’s first open design item rather than as something the run supports.

Revising the policy without quietly rewriting it

Two selection failures survived the ladder. Under architectural review the sentence recording that the chapter has no committed run — arguably the single most decision-relevant unit in the corpus — ranked last among preferred classes and lost to the budget (E3). Under implementation review the measurement contract was ruled ineligible as a class the work type does not list (E2).

The tempting repair is to edit the preference table until those cases pass. The book’s own methodology forbids it: a change to policy is a change to a measured variable. So a proposed frame is an immutable version, a backtest replays every task under it, and promotion requires the primary metric to improve with no declared gate breached.

Book result. Four proposals were replayed over the nine memory-book tasks, primary metric must-include recall, baseline 0.880.

• raise chapter prose for architectural review — the direct fix for the E3 failure — rejected: −0.074. It repairs one task and damages several others.
• add the measurement contract to implementation review — promoted: +0.037.
• widen the class pull from three candidates to five — promoted: +0.065.
• all three together — rejected: −0.046, and a precision gate breached. The revisions interact; two individually good changes plus one bad one are worse than either good one alone.

The first line is the useful one. The obvious repair, derived directly from an observed failure and locally correct, makes the system worse overall. A frame edited case by case would have adopted it. This is what separates a policy that adapts from a system that quietly revises what it considers important, and it is why the backtest harness exists before any learned ranking does.

What this does not establish

The circularity risk is real. Goal relevance is computed from a class-preference table inside the project frame, and the ledgers were authored by hand. Three things limit it: one preference table per work type serves every task of that work type, so it cannot encode a particular task’s answer; required units span several classes while units of a preferred class are frequently distractors for a given task, so query relevance still has to do work; and the frame-capture and inferred-frame results measure the case where a ledger-tuned table would flatter itself most. The limits are structural, not complete. The verdict is fixture-level.

The corpus is controlled. Sixty-eight units and eleven tasks over a corpus built for this experiment. The benchmark contract still requires a real-project corpus with manually adjudicated queries, and this chapter has not supplied it.

Bundle ordering is untested. Position inside a long context changes how usable the information is (Lost in the Middle). This layer decides membership and records order, and does not yet treat order as a variable. That belongs to the context-budget work rather than here.

Better context did not produce better answers. The reader stage ran the same model, prompt and temperature across conditions, and mechanical claim scoring gives:

The full builder matches the baseline’s coverage using 22% fewer context tokens and eliminates the answers that assert superseded state. C4 is slightly worse on coverage than C0. Eleven tasks with mechanical scoring cannot carry a behavioural conclusion in either direction, and this one is reported as what it is: no demonstrated answer-quality gain, one demonstrated safety gain, and a cost reduction.

The verdict

Against the outcomes declared before the run, this is a type C result: context selection improves substantially and measurably, and the downstream behaviour does not follow it.

Context quality improves and the improvement is specific. A declared work frame moves must-include recall by 0.13 and precision by 0.27 over strong retrieval at a matched budget, and the same query under two objectives produces bundles sharing one unit instead of twenty-one. Project scoping earns a governance role, exactly as the Chapter 6 Nexus did. The Chapter 7, 8 and 9 signals earn a narrow safety role — harmful admission and superseded-state assertions both to zero — and not a broad quality one. Coalescing and adjacency have not earned their place at all on this corpus.

Three qualifications keep the result honest, and each of them matters more than the headline. The gains hold with a frame a person wrote; with a frame the model inferred, they nearly vanish. Answer quality did not improve, so the demonstrated benefits are cost and safety, not correctness. And all of it is fixture-level, on a corpus built for the purpose.

So the layer is earned, unevenly, and conditionally. The sentence that survives is the chapter’s whole argument, and it has to carry its own qualification:

What matters from memory depends on the work being done. An explicit work frame can substantially improve context selection over query-only retrieval — but because an incorrect frame excludes the very evidence the system needs, frame conditioning has to be observable, calibrated and reversible.

Stated as the run supports it: on the controlled fixture, frame-conditioned context improved required-evidence recall, eliminated cross-project leakage, and — with the temporal and evidential filters — removed harmful stale evidence. It did not improve answer correctness over strong retrieval, but matched it on less context. Automatically inferred frames nearly erased the gain.

What we still do not know

The clearest unfinished business is sitting in the results table, in a column the discussion has so far passed over. The oracle reaches perfect required-evidence recall and perfect precision on a mean of 587 estimated tokens. The best real condition spends 1,161 to reach 0.901 and 0.464. Roughly half the admitted budget, under a correct frame and a good candidate pool, is still spent on evidence the work does not need.

That gap is a better next question than anything this chapter speculated about, because it is already measured and the ceiling is already known:

Given a correct frame and a high-recall candidate pool, how close can context construction get to the minimum sufficient set under a hard budget?

Selection decides membership; it has not yet been asked to be economical about it. Ordering is untested, compaction did not pay, and the oracle shows there is roughly a factor of two available.

Two further things remain open and neither is ready. The frame has to be established safely, and the inferred-frame numbers say that is unsolved rather than merely imperfect. And every obligation in this book so far was stated by someone — the migration also broke assumptions nobody wrote down, in fixtures, documentation and configuration that no session mentions and no frame can prefer, because there is no record of them to prefer. A selection policy can only choose among memories that exist.

Research foundations

The premise that similarity is the wrong admission criterion on its own is now independently established. MemGuide retrieves memory aligned to the current dialogue goal and reranks by slot-completion benefit, reporting task-success gains on its own MS-TOD benchmark (author-reported). CDMem conditions retrieval on task and environment through graph-structured indexing over multi-level encodings. MemGate states the sharper form — a semantically related memory can be contextually inappropriate — and treats long-term memory as a control channel, which is the framing this chapter’s frame-capture measurement inherits. The write-side counterpart is TaskMem, which frames what to memorise as a learnable task-conditioned policy; it is deliberately outside this chapter, whose policy governs recall rather than retention.

For the budget itself, Anthropic’s context-engineering guidance treats context as a finite attention budget to be curated, with just-in-time retrieval and compaction that maximises recall before precision (vendor-authored, labelled as such). Generative Agents supplies the precedent for separating relevance, recency and importance rather than fusing them into one similarity score; this chapter keeps the separation, makes the importance term conditional on a declared frame, and requires every component to remain inspectable. MemGPT supplies the framing of limited context as a resource to be managed rather than a constraint to be suffered. Lost in the Middle marks the boundary this chapter does not cross.

What appears to be new is the combination rather than any single mechanism: a durable project frame separated from an ephemeral work frame built from arbitrary signals, frames that participate in candidate generation as well as reranking, an exact frozen bundle, a candidate-by-candidate selection trace treated as a first-class artifact, and policy revision by replay under declared regression gates. The claim is offered cautiously — the individual parts all have precedents, and the review was not exhaustive.

References

• Yiming Du and colleagues, MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agents (2026).
• Pengyu Gao and colleagues, An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025).
• Jiawen Zhang and colleagues, Beyond Similarity: Trustworthy Memory Search for Personal AI Agents (2026).
• Tao Zou and colleagues, Task-Focused Memorization for Multimodal Agents (2026).
• Prithvi Rajasekaran and colleagues, Effective context engineering for AI agents (Anthropic, 2025).
• Park and colleagues, Generative Agents: Interactive Simulacra of Human Behavior (2023).
• Charles Packer and colleagues, MemGPT: Towards LLMs as Operating Systems (2023).
• Nelson F. Liu and colleagues, Lost in the Middle: How Language Models Use Long Contexts (2023).