The Memory Contamination Problem

Chapter 13 ended with a strict recovery invariant:

repair
→ new candidate state
→ re-measure
→ re-authorize

and with one additional rule:

candidate in HOLD
→ do not persist as trusted factual memory

That second rule changes the scale of the problem.

A hallucinated sentence displayed once is transient.

The same sentence written into persistent state can survive the conversation that created it.

It can then be:

retrieved
summarized
copied
cited
used as agent experience
used to fill a user profile
inserted into a vector index
fed into another model
used as a future repair source

and eventually appear to the system as if it came from somewhere else.

The failure has changed form.

It is no longer merely:

model generated unsupported claim

It has become:

system state contains unsupported influence

That is the subject of this chapter.

Persistent memory is a future influence channel. In factual systems, it can become a future evidence channel.

A contamination failure across two sessions

Consider the claim from Chapter 13:

Q3 revenue was approximately $46 million.

The available evidence contains no Q3 revenue value.

The correct verification state is:

INSUFFICIENT_EVIDENCE

Now imagine a naive memory writer that stores every model-generated statement that looks useful.

Session 1

candidate = "Q3 revenue was approximately $46 million."
verification = INSUFFICIENT_EVIDENCE
response commitment = HOLD

A flawed writer nevertheless produces:

memory m1
content   = "Q3 revenue was approximately $46 million."
origin    = MODEL_OUTPUT
lifecycle = ACTIVE

Session 2

The user asks:

What was Q3 revenue?

A similarity retriever returns m1.

The prompt now contains:

Relevant memory:
Q3 revenue was approximately $46 million.

The generator answers:

Q3 revenue was approximately $46 million.

No external source has entered the system.

The unsupported assertion has merely changed roles:

    graph LR
    S1[SESSION 1 MODEL OUTPUT] --> PM[PERSISTENT MEMORY]
    PM --> S2[SESSION 2 RETRIEVED CONTEXT]
    S2 --> S2O[SESSION 2 MODEL OUTPUT]
  

That role change can create a false appearance of evidence.

To make the admission failure concrete, I built a deterministic reference example.

Environment

Python 3.13.5
standard library only
no external model calls

The observed output was:

FLAWED_STORE
m1 origin=MODEL_OUTPUT verification=UNVERIFIED lifecycle=ACTIVE
flawed factual retrieval: ['m1']

GOVERNED_STORE
m1 origin=MODEL_OUTPUT verification=UNVERIFIED lifecycle=QUARANTINED
m2 origin=EXTERNAL_SOURCE verification=VERIFIED lifecycle=ACTIVE
m3 origin=MEMORY_DERIVED verification=UNVERIFIED lifecycle=QUARANTINED parents=['m1', 'm2']
factual-evidence retrieval: ['m2']
general recall: ['m2']

The important line is m3.

It is a derived summary built from one verified memory and one unverified memory.

The summary does not become verified merely because a new transformation rewrote the inputs into cleaner prose.

Transformation does not upgrade provenance.

The controller example proves that invariant under a deterministic structured oracle.

It does not establish the accuracy of real-world memory extraction, verification, or summarization.

Where we are

The reliability architecture now has three distinct boundaries.

Response commitment

candidate
→ measurement
→ policy
→ may this leave the system as an assertion?

Memory admission

memory candidate
→ admission policy
→ may this become persistent state, with which capabilities?

Memory use

stored memory
→ access + admissibility policy
→ may this influence this purpose?

These boundaries are related.

They are not interchangeable.

A response that received:

commitment = PERMIT

is only eligible for memory admission evaluation.

It is not automatically admitted.

Likewise, a memory that was legitimately persisted is not automatically admissible for every future task.

This gives the chapter’s first type discipline:

1. Persistence changes the failure class

Without persistence:

bad generation
→ bad response
→ session ends

With persistence:

bad generation
→ stored state
→ future retrieval
→ future reasoning
→ future output
→ possible re-storage

The first is primarily an output error.

The second is a state corruption process.

Persistent writable memory introduces:

persistence
statefulness
propagation

A 2026 survey of long-term-memory security in LLM agents uses those properties to explain why cross-session writable memory creates a qualitatively different attack surface.[1]

The book’s concern is broader than adversarial attacks.

The same state can be contaminated by:

hallucinated facts
stale facts
misresolved identities
incorrect summaries
failed agent trajectories
unverified user assertions
bad retrieval results
repair candidates that never passed policy

The database cannot tell whether a false value came from:

a confused model

or:

a malicious injection

unless the system preserved origin and admission evidence.

So accidental contamination and adversarial poisoning meet at the same architectural boundary:

WRITE

Their threat models differ.

Their need for governed writes does not.

2. Memory is typed state, not one epistemic object

The word memory hides several very different objects.

interaction event
user assertion
user preference
tool observation
external-source claim
model inference
derived summary
agent procedure
failure experience

These objects answer different questions.

Consider:

User said: "My manager is Alice."

There are at least two possible memory propositions.

Assertion event

At time t, the user stated:
"My manager is Alice."

The system may have directly observed that event.

Asserted world proposition

manager(user) = Alice

That proposition is still merely user-asserted unless another admissible source establishes it.

Therefore:

git status returned a clean working tree at 18:32

may be a verified runtime observation.

It does not establish:

the repository is clean forever.

Memory should preserve what was actually observed, asserted, inferred, or derived.

3. Split origin, verification, time, lifecycle, and capabilities

The first draft of this chapter used one broad trust enum containing values such as:

VERIFIED
USER_ASSERTED
TOOL_OBSERVED
MODEL_DERIVED
STALE
REVOKED

That mixes different dimensions.

A stronger memory record separates them.

Origin

Where did this representation come from?

USER
TOOL
EXTERNAL_SOURCE
MODEL_OUTPUT
MEMORY_DERIVED

Verification state

What is established about the proposition?

VERIFIED
UNVERIFIED
INSUFFICIENT_EVIDENCE
REFUTED
CONFLICTING
UNRESOLVED
NOT_VERIFIABLE

Temporal validity

When is the proposition supposed to hold?

valid_from
valid_to

System time

When did the system observe and store it?

observed_at
recorded_at
superseded_at

Lifecycle

What is its persistence state?

EPHEMERAL
QUARANTINED
PROVISIONAL
ACTIVE
ARCHIVED
SUPERSEDED
REVOKED
NEEDS_REVALIDATION

Capabilities

For which purposes may it be consumed?

PERSONALIZATION
CONVERSATIONAL_CONTEXT
FACTUAL_EVIDENCE
AUTONOMOUS_ACTION_JUSTIFICATION
REPAIR_PLANNING
REGRESSION_TEST
PROCEDURE_IMITATION

Now the system can ask different questions independently:

Where did this come from?
Is its proposition established?
Is it still current?
May it still be retained?
May it be used for this purpose?

One enum should not answer all five.

4. Memory admission is its own policy decision

A memory candidate should cross a write gate just as a generated answer crosses an acceptance gate.

A typed record might look conceptually like:

memory_candidate = {
    "memory_id": "mem_1842",
    "content": "Q3 revenue was approximately $46 million.",
    "memory_type": "FACTUAL_CLAIM",

    "origin_kind": "MODEL_OUTPUT",
    "verification_state": "INSUFFICIENT_EVIDENCE",
    "lifecycle_state": "EPHEMERAL",

    "parent_memory_ids": [],
    "support_claim_ids": [],
    "evidence_ids": ["filing_q1", "filing_q2"],
    "verification_record_id": "verify_1842",

    "requested_uses": ["FACTUAL_EVIDENCE"],

    "valid_from": None,
    "valid_to": None,
    "observed_at": "2026-08-30T18:39:00+01:00",

    "retention_scope": "project",
}

Admission then decides capabilities, not one global trusted bit.

For example:

admission_decision = {
    "persist": True,
    "lifecycle_state": "QUARANTINED",
    "allowed_uses": ["REGRESSION_TEST"],
    "forbidden_uses": [
        "FACTUAL_EVIDENCE",
        "AUTONOMOUS_ACTION_JUSTIFICATION",
    ],
    "policy_version": "memory-admission-2026.08.30.3",
}

Recent work on Adaptive Memory Admission Control similarly treats long-term-memory admission as an explicit structured decision rather than automatic retention or purely opaque LLM judgment.[2]

One important qualification follows from our architecture:

Future utility and epistemic admissibility are different objectives.

A recurring model misconception may be extremely useful as a regression artifact while remaining forbidden as factual evidence.

5. Derivation carries provenance obligations

Suppose:

m1 = unverified rumor
m2 = verified Q2 revenue

and a summarizer creates:

m3 = summary(m1, m2)

It is tempting to assign one scalar trust score to each node and define:

USER_ASSERTED, TOOL_OBSERVED, VERIFIED, and MODEL_DERIVED are not naturally one total numerical order for every purpose.

A better rule is capability and lineage preserving.

For each factual claim in the child, record which parent claims are actually required to support it.

For example:

summary_claim = {
    "claim_id": "m3:c2",
    "content": "Q2 revenue was $43.8M.",
    "required_parent_claims": ["m2:c4"],
    "derivation": "SUMMARIZE",
}

If another summary claim depends on both m1 and m2, the unresolved provenance obligation from m1 survives into that claim.

So:

A derived claim carries the unresolved provenance obligations of every required support path until new admissible verification resolves them.

The entire document need not inherit taint from an unrelated parent that merely appeared in the summarizer context.

That is why claim-level lineage matters.

A derivation cannot create stronger factual evidence than its required sources contain.

6. Repetition is not corroboration

Suppose the same unverified claim is stored in three forms:

m1:
Q3 revenue was approximately $46M.

m2:
Revenue reached roughly $46M in Q3.

m3:
Third-quarter revenue was about forty-six million dollars.

A vector retriever may return all three.

The generator sees:

three agreeing passages

but lineage reveals:

cand_v1
   ├── m1
   ├── m2
   └── m3

This is:

3 texts
1 evidential lineage

But even root count is not enough.

Suppose the system sees:

news_site_A
news_site_B
blog_C

All three may independently exist as memory roots while ultimately copying one press release or one original error.

Therefore:

DERIVED_FROM
COPIED_FROM
SYNDICATED_FROM
SHARES_PRIMARY_SOURCE_WITH
SOURCE_FAMILY

At retrieval time, corroboration should be computed over independent admissible source families, not chunk count.

Repeated text is not independent evidence.

7. Derivation lineage and evidential support are different graphs

A versioned derivation graph usually moves forward in time:

cand_v1
→ mem_v1
→ summary_v1
→ cand_v2
→ mem_v2

That graph can remain a DAG.

But the support graph asks another question:

Which proposition is being used to justify which proposition?

A self-confirming system can have circular support even if its immutable derivation history is acyclic.

For example:

model invents X
→ stores X
→ later retrieves X
→ cites retrieved X as support for X

Derivation history:

cand_v1 → mem_v1 → cand_v2

Support relation:

X is justified by a representation descended from X itself

That is circular support.

The correct question is:

Where do the support paths terminate?

If all admissible-looking support ultimately descends from the proposition being justified, no independent verification occurred.

A claim cannot verify itself by taking a trip through a vector database.

8. The read path needs access control before relevance and admissibility after relevance

A reliability-aware memory read should not be:

query
→ top-k cosine similarity
→ prompt

It needs at least three stages.

    graph TD
    Q[QUERY + PURPOSE + CALLER] --> HA[HARD ACCESS / ELIGIBILITY FILTER]
    HA --> SR[SEMANTIC RETRIEVAL]
    SR --> EA[CONTEXTUAL / EVIDENTIAL ADMISSIBILITY]
    EA --> UC[USABLE CONTEXT]
  

Memory retrieval is governed before and after similarity search, so a relevant memory is still excluded if it lacks the rights or evidential status required for the current use.

Pre-retrieval eligibility

Exclude memories that should not even enter the retrieval candidate set because of:

tenant / user boundary
sensitivity
lifecycle state
caller authorization
hard purpose restriction
legal deletion / retention state

Semantic retrieval

Rank eligible memories for relevance.

Post-retrieval admissibility

Evaluate:

verification state
freshness
conflict
scope
source-family independence
task risk
requested purpose

This matters operationally because an item can leak before final prompt construction if a reranker, log, or remote retrieval service sees it.

For a simple PostgreSQL + pgvector deployment, some of the first gate can be ordinary metadata filtering:

SELECT *
FROM memory
WHERE lifecycle_state = 'ACTIVE'
  AND allow_factual_evidence = TRUE
  AND tenant_id = $1
ORDER BY embedding <-> $2
LIMIT 10;

pgvector supports nearest-neighbor queries with ordinary SQL WHERE filters, although approximate indexes require care because filtering can occur after index scanning and may affect recall.[10]

This is enough for some applications.

It is not enough for all of them.

Lineage, conflict, supersession, claim dependencies, and source-family relations are graph-shaped state. A graph-backed or hybrid graph-plus-vector architecture can represent those relations explicitly while vector search handles semantic candidate retrieval.

Microsoft’s GraphRAG is one concrete research implementation: its indexing pipeline extracts entities, relationships, claims, community structure, and embeddings.[11]

It should not be treated as a universal industry standard—the Microsoft repository itself describes the project as research-oriented and, as of 2026, largely in maintenance mode.[11]

The architectural point is broader:

vector search
→ relevance

graph / metadata / policy
→ epistemic structure and admissibility

9. Context serialization should expose restrictions without dumping the database

Rich memory records may contain:

lineage
source families
verification records
validity intervals
conflict edges
policy versions

The model usually does not need every field verbatim in its context window.

Dumping the entire memory schema into the prompt can add latency, token cost, and distraction.

A better read layer serializes the minimum decision-relevant metadata.

For an ordinary verified memory:

Q2 revenue was $43.8M. [source: filing_q2]

For an anomalous memory:

<memory state="CONFLICTING" use="CONTEXT_ONLY">
Sources disagree about the current CEO.
</memory>

For a user assertion:

<memory origin="USER" verification="UNVERIFIED" use="PERSONALIZATION">
User stated that their manager is Alice.
</memory>

The full provenance record remains outside the model and can be dereferenced when policy or verification needs it.

Prompt context should expose semantics that affect model behavior, not mirror the entire persistence schema.

10. Factual memory should often point back to evidence

A verified memory should not become authoritative merely because the system copied a fact into its own store.

Where possible, factual memory should act as an index into durable evidence.

Instead of storing only:

Q2 revenue = $43.8M
verification = VERIFIED

retain:

normalized claim
verification state
source ID
source version
exact support span
evidence snapshot reference
retrieval / verification time
valid_from / valid_to

Then a high-risk future action can use:

memory hit
→ dereference original evidence
→ verify source still admissible / current
→ use source as evidence

This preserves a crucial distinction:

memory helps find evidence

versus:

memory has become the evidence simply by being stored.

11. Summaries and compaction need claim-level preservation contracts

Long-running systems summarize memory to control context size.

That creates a trust-laundering surface.

Suppose raw memory says:

Source A reports X.
Source B disputes X.
verification = CONFLICTING

A bad summarizer writes:

X is true.

Two failures occurred.

Epistemic elevation

CONFLICTING
→ VERIFIED-looking assertion

Lineage collapse

A + B
→ summary with source identities lost

So compaction should preserve, at claim level where relevant:

required parent claims
verification state
source families
conflict state
uncertainty
validity interval
scope

A safe summary might instead say:

Sources disagree about X.

and preserve links to both claims.

The contract is:

Compression may reduce tokens. It must not silently increase certainty or erase the lineage required to interpret the claim.

Recent 2026 work on MemGuard takes a related lifecycle approach by persisting verifier outputs and reusing them during memory admission, retrieval, conflict handling, summarization, and archival.[6]

12. Time and conflict need resolved semantics

Persistent memory needs at least two time dimensions. This is standard bitemporal modelling — the valid-time / transaction-time split from Snodgrass, carried into SQL:2011 as system-versioned tables.[14]

Valid time

When was the proposition true in the world?

valid_from
valid_to

System time

When did the system learn and store it? (Bitemporal databases call this transaction time; SQL:2011 calls it system time.)

observed_at
recorded_at
superseded_at

For example:

CEO = Bob
valid_from       = 2026-07-01
source_published = 2026-07-03
system_recorded  = 2026-07-05

Now two questions can be answered differently:

Who was CEO on June 20?

and:

What did the system believe on July 2?

Conflict detection also needs resolved identity, relation, time, and scope.

These are not necessarily contradictory:

Alice was CEO in 2025.
Bob was CEO in 2026.

Nor are:

CEO of US parent = Alice
CEO of European subsidiary = Bob

So before creating:

m1 CONFLICTS_WITH m2

resolve:

entity
relation
scope
validity interval

Memory conflict is a structured relation, not merely text that appears opposite.

13. Revocation invalidates descendants; it does not blindly delete them

Suppose memory m1 later becomes invalid.

And suppose m1 contributed to:

m3 summary
m7 recommendation
m9 profile entry
m12 graph edge

Changing only m1 is not enough.

But blindly revoking every descendant is also wrong.

A descendant may have independent support.

For example:

m3 depends on:
    m1
    authoritative m2

If m1 is revoked, m2 may still establish the proposition.

Therefore:

    graph TD
    REV[REVOKE / CHANGE ROOT] --> FIND[find dependent claims]
    FIND --> MARK[mark NEEDS_REVALIDATION]
    MARK --> RECOMP[recompute support from remaining admissible roots]
    RECOMP --> STATE[ACTIVE / QUARANTINED / REVOKED]
  

This is ordinary incremental-build invalidation applied to epistemic state.

The system also needs impact analysis for artifacts that have already escaped memory:

responses
tool actions
database writes
reports
notifications
external decisions

Some consequences cannot be rolled back by changing memory.

They may require:

correction
notification
compensating action
reissue
human escalation

14. Epistemic revocation and privacy erasure are different operations

For epistemic correction, versioned history is often valuable:

old assertion
→ superseded / revoked
→ corrected version

That supports replay and audit.

But personal-data governance can impose different requirements.

The GDPR’s Article 17 provides a right to erasure in specified circumstances and requires erasure without undue delay when applicable, subject to its conditions and exceptions.[12]

California’s CCPA likewise provides consumers a right to delete personal information collected from them, subject to exceptions.[13]

Therefore:

epistemic correction
≠
privacy erasure

A privacy deletion workflow may need to:

remove or irreversibly anonymize content
remove embeddings / replicas where required
invalidate descendants that contain the data
propagate deletion to caches and derived stores
stop future retrieval

while retaining only whatever non-content tombstone or audit metadata is lawfully necessary.

The exact legal obligation depends on jurisdiction, purpose, exemptions, and the application’s role; this chapter is not legal advice.

The systems point is narrow and important:

Do not make “we preserve all history forever” an invariant of the memory architecture.

The same lineage machinery used for epistemic rollback is also what makes targeted erasure technically possible.

15. Failure and experience memory need environment fingerprints

Agents increasingly store:

plans
procedures
tool sequences
successful trajectories
failed trajectories

These memories can influence behavior even when they are not factual propositions.

A failure should be stored as a failure:

failure_memory = {
    "memory_type": "FAILURE_EXPERIENCE",
    "outcome": "FAILED",
    "allowed_uses": ["REGRESSION_TEST", "REPAIR_PLANNING"],
    "forbidden_uses": ["SUCCESSFUL_PROCEDURE_IMITATION"],
}

A successful trajectory also needs conditions under which success was established:

repo commit
tool versions
OS/runtime
permissions
inputs
preconditions
postconditions
verifier result

Otherwise:

This command sequence solved the issue.

can be retrieved into a completely different environment and treated as reusable policy.

MemoryGraft is important here because its attack surface is precisely successful-looking experience memory: poisoned trajectories can later be retrieved and imitated on semantically similar tasks.[4]

So procedural memory needs:

success + the environment and constraints under which success was established.

16. Accidental contamination and adversarial poisoning require overlapping but different controls

The same write gate protects against both accidental and malicious persistence.

But the threat models differ.

RevPRAG studies poisoned RAG databases and detection of poisoned responses.[3]

MemoryGraft studies persistent compromise through poisoned experience retrieval.[4]

Hidden in Memory studies delayed poisoning in which adversarial external context induces fabricated persistent memories that later steer stateful assistants.[5]

These are security attacks.

The chapter’s broader reliability lesson is:

The memory store should not need to know whether bad state was malicious before it applies basic epistemic controls.

17. Evaluate contamination across the lifecycle

A useful benchmark should be organized by where the failure enters or escapes.

Two especially useful operational metrics are:

Taint-escape rate

How often does a descendant with unresolved contaminated ancestry
incorrectly become admissible as clean factual evidence?

Exposure before containment

How many retrievals, responses, or actions consume contaminated state
before quarantine or revocation occurs?

Also report:

descendant count per contaminated root
maximum propagation depth
post-revocation exposure
false invalidation rate

One memory-quality scalar would hide these failure modes.

Event-driven plus periodic maintenance

Do not rely only on scheduled scans.

Trigger immediate invalidation when:

source revoked
source updated
parent memory revoked
user correction arrives
entity merge changes resolution
policy changes
schema changes

Then run periodic scans for:

staleness
poisoning patterns
missed conflicts
long-lived high-impact memories

CAMS proposes a lifecycle security architecture containing guarded writes, provenance, temporal monitoring, graph analysis, periodic scanning, and retroactive quarantine.[7]

18. A production memory architecture

The chapter’s canonical architecture now has one write gate and two read gates.

Write side

    graph TD
    US[USER / TOOL / SOURCE / MODEL] --> MC[MEMORY CANDIDATE]
    MC --> OVT[ORIGIN + VERIFICATION + TIME]
    OVT --> CL[CLAIM-LEVEL LINEAGE]
    CL --> AP[ADMISSION POLICY]
    AP --> PERS[PERSISTED WITH CAPABILITIES]
  

The write gate turns memory creation into an authorization step: only candidates with origin, verification state, time, and lineage can receive durable use capabilities.

Read side

    graph TD
    Q[QUERY + PURPOSE + CALLER] --> AE[ACCESS ELIGIBILITY]
    AE --> RR[RELEVANCE RETRIEVAL]
    RR --> EA[EVIDENTIAL ADMISSIBILITY]
    EA --> CS[CONTEXT SERIALIZER]
    CS --> GA[GENERATION / ACTION]
  

Every generated output that might be remembered returns to the write side.

This gives three different questions:

May we store it?
May this caller retrieve it?
May it be used for this purpose?

Those questions should never be collapsed into:

is this vector in the database?

Memory ledger

Store enough metadata to replay how the memory acquired its current capabilities:

memory_id
content / immutable content reference
memory_type
origin_kind
verification_state
verification_record_id
valid_from / valid_to
observed_at / recorded_at
lifecycle_state
parent claim IDs
source-family IDs
allowed uses
forbidden uses
admission_policy_version
memory_schema_version
verification_schema_version
embedding_model_version
normalizer_version
entity_resolution_version
evidence snapshot reference + hash

Versioning the schema and resolvers matters because a later entity-resolution or policy change can alter what historical records mean.

19. Counterfactual memory tests

The metamorphic discipline from earlier chapters applies directly.

Same content, different origin

Hold text fixed:

EXTERNAL_SOURCE + VERIFIED
→ MODEL_OUTPUT + UNVERIFIED

Expected:

factual admissibility may decrease.

The system must respond to epistemic metadata, not only text.

Derivation laundering

    graph LR
    m1[verification=UNVERIFIED] --> m2[summarize] --> m3[summarize] --> m4[summarize]
  

Summarization may change wording, but it must not launder an unverified memory into a fact with stronger evidential rights.

Expected:

m4 remains lineage-linked to m1
m4 does not acquire factual-evidence capability

Source-family duplication

Create five paraphrases and three URLs that all descend from one primary source.

Expected:

chunk count > source-family count
corroboration uses source-family count

Revocation

Revoke one required root.

Expected:

descendants become NEEDS_REVALIDATION
not blindly all REVOKED

Staleness

Advance the requested as-of date beyond valid_to.

Expected:

memory loses current-state authority.

Response permission

Change:

response commitment HOLD
→ PERMIT

Expected:

HOLD cannot enter active factual memory
PERMIT becomes eligible for admission policy
PERMIT does not auto-admit

20. Using AI: extract memory candidates, do not certify them

An LLM can help turn conversation into structured memory candidates.

A weak instruction is:

Remember the important facts from this conversation.

That lets the model decide:

what is important
what is factual
what is persistent
what is verified
what may be reused

A stronger instruction is:

Extract candidate memory units only.

For each candidate return:
- exact source span
- proposition_vs_observation
- memory type
- normalized content
- origin kind
- temporal scope
- entities involved
- possible parent claims
- requested future use

Do not mark anything VERIFIED.
Do not decide persistence.
Do not remove uncertainty or conflict.

For:

My manager is Alice.

AI may propose two objects:

ASSERTION_EVENT
"User stated that their manager is Alice."
origin = USER
verification = VERIFIED_OBSERVATION_OF_EVENT

and:

WORLD_PROPOSITION
"User's manager is Alice."
origin = USER
verification = UNVERIFIED

The system—not the extraction model—decides admission and future capabilities.

A common extraction failure is:

User mentioned X
→ extractor marks X VERIFIED

That should fail schema/policy validation.

The model observed an assertion.

It did not verify the proposition asserted.

What you should now be able to answer

After this chapter, you should be able to explain:

1. Why persistent hallucination is a state-corruption problem rather than only an output error.
2. Why origin, verification, temporal validity, lifecycle, and use capabilities must be separate fields.
3. Why a verified assertion event is not the same thing as a verified world proposition.
4. Why response authorization does not imply memory admission.
5. Why transformation and repetition cannot manufacture independent evidence.
6. Why claim-level lineage and source families matter for corroboration.
7. Why derivation history and evidential support are different graphs.
8. Why retrieval needs access filtering before similarity search and admissibility after it.
9. Why factual memory should often dereference original evidence for high-risk use.
10. How compaction, time, conflict, revocation, privacy erasure, and procedural memory change the governance problem.

Exercises

Exercise 1 — Build a typed memory gate

Create four candidates:

verified tool observation
user-stated preference
model-generated factual inference
failed repair candidate

For each, assign:

origin
verification state
temporal scope
lifecycle
allowed uses

Confirm that the model inference cannot become factual evidence merely through admission.

Exercise 2 — Test trust laundering

Create:

m1 origin=MODEL_OUTPUT verification=UNVERIFIED

Then execute three summarization steps:

m1 → m2 → m3 → m4

Write a unit test asserting:

m4 remains connected to m1 through claim-level lineage
FACTUAL_EVIDENCE not in m4.allowed_uses

Then add a new admissible external source that independently verifies the proposition and test that policy may explicitly upgrade the new version.

Exercise 3 — Test source-family corroboration

Create five stored chunks from three URLs, all syndicated from one primary source.

Verify:

chunk_count = 5
url_count = 3
independent_source_family_count = 1

Exercise 4 — Revoke and revalidate

Build:

m1 → m3 → m7
m2 → m3

where m2 independently supports m3.

Revoke m1.

Confirm that:

m3 → NEEDS_REVALIDATION

and is restored if m2 is sufficient, rather than blindly revoked.

Exercise 5 — Relevance versus admissibility

Create a query for which:

highest similarity = QUARANTINED / UNVERIFIED
second similarity  = ACTIVE / VERIFIED

Confirm that a factual query uses the second memory.

Then run a personalization query where the first memory has an allowed personalization capability and show that the result may differ.

Exercise 6 — Assertion event versus proposition

Input:

My manager is Alice.

Store separately:

verified observation of what the user said
unverified world proposition about the manager

Write tests proving they receive different capabilities.

The deeper lesson

A stateless hallucination is visible at the moment it occurs.

A persistent hallucination can become invisible precisely because it survived.

The database begins to confer legitimacy on text simply by containing it.

That is the trap.

stored
≠ verified

active
≠ true

retrieved
≠ admissible

relevant
≠ authorized

repeated
≠ corroborated

root
≠ independent source

summarized
≠ independently supported

response permitted
≠ memory admitted

A memory system must preserve the distinctions the rest of this book worked to build.

Its canonical record therefore looks more like:

MEMORY RECORD
│
├── CONTENT
│
├── ORIGIN
│
├── VERIFICATION
│
├── TEMPORAL VALIDITY
│
├── LIFECYCLE
│
├── CLAIM-LEVEL LINEAGE
│
├── SOURCE FAMILIES
│
└── CAPABILITIES

and the two central policy functions are:

The final chapter can now assemble the whole architecture.

The question is no longer:

How do we stop a language model from ever hallucinating?

That is the wrong systems goal.

The question is:

How do we build a system that assumes its model can fail, detects the failure where possible, limits what survives, and keeps unverified output from becoming action or state?

Related but distinct: training-time feedback

Model-generated data entering future training corpora creates a related but mechanically different feedback loop.

Memory contamination changes agent behavior through inference-time persistent context.

Recursive synthetic-data training can change model weights.

The first can often be mitigated by memory invalidation and revocation within one system.

The second can be distributed across datasets and future model versions and may be much harder to reverse.

Shumailov et al. showed degradation under some recursive model-generated training regimes.[8]

Kazdan et al. later showed that collapse is not inevitable under every data workflow and that the outcome depends strongly on how real and synthetic data are retained and accumulated.[9]

The mechanisms differ.

The shared systems pattern is only:

model output
→ persistent data
→ future model input
→ altered future behavior

That family resemblance is enough for this chapter.

Research roots

1. Zehao Lin, Xixuan Hao, Renyu Fu, Shaobo Cui, Kai Chen, Chunyu Li, Zhiyu Li and Feiyu Xiong, “A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle,” arXiv:2604.16548, 2026. Frames writable persistent memory around lifecycle phases including write, store, retrieve, execute, propagate, and rollback, and emphasizes provenance, versioning, and governance across the lifecycle. https://arxiv.org/abs/2604.16548

2. Guilin Zhang, Wei Jiang, Xiejiashan Wang, Aisha Behr, Kai Zhao, Jeffrey Friedman, Xu Chu and Amine Anoun, “Adaptive Memory Admission Control for LLM Agents,” ICLR 2026. Treats memory admission as an explicit structured decision rather than automatic retention and evaluates an interpretable admission framework on long-term conversational memory. https://arxiv.org/abs/2603.04549

3. Xue Tan, Hao Luan, Mingyu Luo, Xiaoyan Sun, Ping Chen and Jun Dai, “RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis,” Findings of EMNLP 2025, pp. 12999–13011. Studies poisoning of RAG knowledge databases and detection of poisoned responses. https://aclanthology.org/2025.findings-emnlp.698/

4. Saksham Sahai Srivastava and Haoyu He, “MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval,” arXiv:2512.16962, 2025. Studies persistent poisoning of experience memory, where malicious successful-looking trajectories are retrieved and imitated on later tasks. https://arxiv.org/abs/2512.16962

5. Sidharth Pulipaka, Stanislau Hlebik, Leonidas Raghav, Sahar Abdelnabi, Vyas Raina, Ivaxi Sheth and Mario Fritz, “Hidden in Memory: Sleeper Memory Poisoning in LLM Agents,” arXiv:2605.15338, 2026. Studies delayed attacks in which adversarial external context induces fabricated persistent memories that later steer stateful assistants. https://arxiv.org/abs/2605.15338

6. Haoyu Wang, Guangyuan Dong, He Liang, Zijing Zhang, Jiachen Luo, Chuang Liu, Chao Xue and Hao Tang, “MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance,” arXiv:2608.21867, 2026. Treats verifier output as persistent lifecycle metadata used in memory admission, retrieval, conflict handling, summarization and archival. https://arxiv.org/abs/2608.21867

7. “Cognitive Autonomous Memory Security (CAMS) against injection and extraction attacks in long-term memory of AI agents,” Egyptian Informatics Journal 34, 100983, 2026. Proposes a multi-layer memory-security architecture including write guards, provenance, temporal monitoring, graph analysis and periodic re-scanning. https://doi.org/10.1016/j.eij.2026.100983

8. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson and Yarin Gal, “AI models collapse when trained on recursively generated data,” Nature 631, 755–759, 2024. Demonstrates degradation under recursive training on model-generated data. https://doi.org/10.1038/s41586-024-07566-y

9. Joshua Kazdan, Rylan Schaeffer, Apratim Dey, Matthias Gerstgrasser, Rafael Rafailov, David L. Donoho and Sanmi Koyejo, “Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World,” ICML 2025. Shows that recursive synthetic-data behavior depends strongly on the training workflow and on how real and synthetic data are retained and mixed. https://proceedings.mlr.press/v267/kazdan25a.html

10. pgvector Project, “pgvector: Open-source vector similarity search for Postgres,” current documentation. Demonstrates vector nearest-neighbor search combined with ordinary SQL metadata filtering and documents the interaction between filtering and approximate vector indexes. https://github.com/pgvector/pgvector

11. Microsoft Research, “GraphRAG,” current project documentation. Provides a research implementation that extracts entities, relationships and claims, creates graph/community structures and combines those outputs with vector embeddings for retrieval-augmented generation. The project repository describes the implementation as research-oriented and largely in maintenance mode in 2026. https://github.com/microsoft/graphrag

12. European Union, Regulation (EU) 2016/679, General Data Protection Regulation, Article 17, “Right to erasure (‘right to be forgotten’).” Defines circumstances in which a data subject may obtain erasure of personal data, subject to the regulation’s conditions and exceptions. https://eur-lex.europa.eu/eli/reg/2016/679/oj

13. California Department of Justice, “California Consumer Privacy Act (CCPA),” current guidance, updated August 28, 2026. Lists consumer rights including the right to delete personal information collected from them, subject to exceptions. https://oag.ca.gov/privacy/ccpa

14. Richard T. Snodgrass, Developing Time-Oriented Database Applications in SQL, Morgan Kaufmann, 2000; and ISO/IEC 9075:2011 (SQL:2011), which adds application-time-period tables (valid time) and system-versioned tables (system / transaction time). The valid-time vs transaction-time distinction is the standard bitemporal model. https://www2.cs.arizona.edu/~rts/tdbbook.pdf

Next: Building Systems That Distrust Their Models

The book now has almost every component it needs.

GENERATION
↓
MEASUREMENT
↓
ADVERSARIAL EVALUATION
↓
MULTI-AXIS DIAGNOSIS
↓
ANSWERABILITY
↓
POLICY
↓
VERIFICATION / REPAIR
↓
MEMORY ADMISSION
↓
ACCESS + ADMISSIBILITY
↓
PERSISTENT STATE

The final chapter must assemble these pieces into one engineering architecture.

It must answer:

Which components are deterministic?
Which are probabilistic?
Where are the trust boundaries?
What is logged?
What is replayable?
What happens when sensors disagree?
What happens when verification is unavailable?
What may be committed to persistent state?
How does the system recover after failure?

The final question is:

What does an AI system look like when distrust of its own model is a design principle rather than an emergency patch?