Use with AI · Chapter Application

Audit Whether This Agent Memory Is Relevant, Fresh and Trustworthy

Apply Agents From First Principles Step 07 to a real repository and test whether working, semantic and episodic memory improve decisions without injecting stale or weakly grounded state.

Chapter Application Agents From First Principles Agents with memory, retrieval or cached prior runs Advanced
Read the source chapter → View the learning path →

How to use this

  1. Open a repository-aware AI assistant.
  2. Give it access to the repository or files you want reviewed.
  3. Copy the prompt below and run it unchanged first.
  4. Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are auditing the memory system of an AI agent.

Your job is to determine whether stored information actually improves future decisions and whether retrieval can safely distinguish prior evidence from current truth.

Do not equate persistence with useful memory.
Do not equate retrieved text with authoritative state.

Work in this order:

1. Inventory memory classes
Identify where the repository stores:
- current runtime/task state,
- working memory,
- semantic facts,
- episodic run history,
- procedural rules/workflows,
- exact or approximate caches,
- embeddings/vector indexes,
- summaries or compressed history.

For each, record ownership, lifetime, write path and read path.

2. Trace one memory write
For one representative stored item record:
- source event/data,
- original timestamp,
- author/system/model origin,
- confidence/provenance if any,
- scope (user/project/repository/task/global),
- version or state identity,
- transformation/summarization before storage.

3. Trace one retrieval
Follow a future decision that consumes prior memory.
Record:
- retrieval query,
- candidate generation,
- ranking/filtering,
- freshness checks,
- scope checks,
- conflict handling,
- how retrieved memory enters the model or control logic.

4. Audit relevance
- Determine whether retrieved memories demonstrably affect a decision.
- Check for high-volume history that adds context but no decision value.
- Identify retrieval that favors lexical similarity over task relevance.
- Check whether failed prior attempts are distinguishable from successful knowledge.

5. Audit freshness and truth
- Identify memories that can become stale when repository files, remote services, user intent or external state change.
- Check whether authoritative state is re-read when needed.
- Identify cached outputs whose validity depends on model version, prompt, file hash, tool version or environment.
- Check conflict resolution between old memory and current observation.

6. Audit contamination
- Can one user/project/task retrieve another scope's memory?
- Can speculative model output become durable fact without verification?
- Can summaries erase uncertainty or provenance?
- Can a bad prior run poison later decisions?

7. Failure injection
Reason through:
- file changed after memory was stored,
- prior run failed but its summary sounds authoritative,
- same semantic fact has conflicting versions,
- cache key omits an input that changed,
- retrieval returns relevant but obsolete information,
- memory grows until useful evidence is buried.

Output:

A. Memory architecture map
B. Write-path provenance table
C. Retrieval decision trace
D. Freshness/staleness risks
E. Scope and contamination risks
F. Cache-binding gaps
G. Smallest corrections
H. Tests that demonstrate memory improves decisions safely

Classify each finding as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

The standard is not 'the agent remembers more.'
The standard is: prior information changes a future decision for the better without replacing authoritative current state.