AI Agent Forgets Previous Work? Add Working, Semantic and Episodic Memory

An agent can use the right model, call the right tools, execute the right plan, and still behave as if nothing that happened five minutes ago matters.

You see the symptoms quickly:

• it re-reads files it already inspected;
• it repeats research it already completed;
• it asks for information the user already supplied;
• it forgets why a previous approach failed;
• it loses decisions made earlier in a long task;
• it treats every new run as if the system has never seen the problem before;
• it retrieves an old answer and treats it as current truth;
• it fills the prompt with so much history that the useful information is buried.

The usual response is:

Add memory.

That advice is too vague to be useful.

There is no single thing called agent memory.

A production agent usually needs several different mechanisms that solve different problems:

current task state
      ↓
working memory
      ↓
long-lived records
      ↓
retrieval
      ↓
relevant prior experience

And those mechanisms should not be confused with one another.

This post will build them from first principles.

We will separate:

• runtime state — what is true about the task right now;
• working memory — information useful during the current run;
• semantic memory — reusable facts and knowledge;
• episodic memory — records of previous attempts and outcomes;
• procedural memory — reusable strategies, rules, and workflows;
• cache — previously computed outputs that can be reused exactly or approximately;
• retrieval — the mechanism that decides which stored information is relevant now.

The key idea is simple:

Memory is useful only when past information changes a future decision.

If storing more information does not improve future decisions, you have built a database, not useful agent memory.

Where we are in the series

So far we have built increasingly capable agent machinery.

00  agent loop
      ↓
01  structured actions
      ↓
02  Best-of-N
      ↓
03  critique + revision
      ↓
04  planning + execution
      ↓
05  state + stopping conditions
      ↓
06  tools + routing
      ↓
07  memory

Notice that memory comes after state, planning, loops, and tools.

That ordering matters.

If your agent does not know what step it is currently on, adding a vector database will not fix it.

If your agent has no reliable completion condition, retrieving old runs will not fix it.

If your tool interface is ambiguous, remembering previous tool calls will not fix it.

Memory should extend a functioning runtime, not compensate for an undefined one.

First: state is not memory

Suppose we are building a coding agent.

It is fixing a failing test.

The current task might contain:

state = {
    "goal": "make tests/test_parser.py pass",
    "current_file": "src/parser.py",
    "failing_tests": ["test_nested_block"],
    "attempt": 3,
    "last_tool": "run_tests",
    "last_result": "1 failed, 42 passed",
}

That is runtime state.

It describes what is true now.

It should be explicit, deterministic, and directly inspectable by the runtime.

It should not need semantic retrieval.

It should not be inferred from 150 chat messages.

It should not be reconstructed by asking the model:

What do you think we have already done?

State is the source of truth for the current execution.

A better mental model is:

state = present
memory = relevant past

The agent may use both.

But they are not interchangeable.

Why conversation history is a poor state store

One common design is to keep appending messages:

messages.append({"role": "assistant", "content": output})
messages.append({"role": "tool", "content": tool_result})

Then the entire history is sent back to the model.

That can work for short tasks.

But eventually you get this:

message 1
message 2
message 3
...
message 87
message 88
message 89

Somewhere inside those messages is the fact that:

migration 014 already ran successfully

But the model has to rediscover that fact from prose.

That is a bad runtime contract.

If something is important enough to control execution, promote it into structured state:

state["completed_migrations"] = {14}

Now the runtime can enforce it.

This gives us our first rule:

Do not use long conversation history as a substitute for explicit runtime state.

What working memory actually means

Working memory is information that is useful for the current task but does not necessarily need to survive forever.

For a coding agent this might be:

working_memory = {
    "files_read": {
        "src/parser.py": "...",
        "tests/test_parser.py": "...",
    },
    "recent_errors": [
        "IndexError in parse_nested_block",
    ],
    "current_hypothesis": "parser drops closing delimiter",
}

For a research agent:

working_memory = {
    "claims": [],
    "sources": [],
    "open_questions": [],
    "rejected_sources": [],
}

For a browser agent:

working_memory = {
    "visited_urls": set(),
    "submitted_forms": set(),
    "current_page": None,
    "required_fields": {},
}

This memory is usually bounded by the run.

It reduces repeated work.

It can be summarized, compacted, or discarded at the end.

A tiny working-memory implementation

You do not need a vector database to start.

from dataclasses import dataclass, field
from typing import Any


@dataclass
class WorkingMemory:
    values: dict[str, Any] = field(default_factory=dict)

    def put(self, key: str, value: Any) -> None:
        self.values[key] = value

    def get(self, key: str, default=None):
        return self.values.get(key, default)

    def has(self, key: str) -> bool:
        return key in self.values

    def delete(self, key: str) -> None:
        self.values.pop(key, None)

That looks trivial.

Good.

Start with trivial.

Memory becomes useful because of the decisions it prevents you from repeating.

For example:

if memory.has(f"file:{path}"):
    contents = memory.get(f"file:{path}")
else:
    contents = read_file(path)
    memory.put(f"file:{path}", contents)

That is already agent memory.

It avoids a redundant tool call.

Persisted task state

Some tasks span process restarts.

A deployment agent may be interrupted after step 6 of 12.

A research agent may run for hours.

A coding agent may continue tomorrow.

In those cases, current task state itself may need persistence.

import json
from pathlib import Path


def save_state(path: Path, state: dict) -> None:
    path.write_text(json.dumps(state, indent=2))


def load_state(path: Path) -> dict:
    if not path.exists():
        return {}
    return json.loads(path.read_text())

This is not semantic memory.

It is durable state.

That distinction matters because retrieval rules are different.

If the task state says:

{
  "payment_sent": true
}

you should not retrieve the most semantically similar record.

You should load the exact task record.

Semantic memory

Now consider a different problem.

A coding agent receives this failure:

SQLite database is locked

Months earlier, another run discovered that the project uses a long transaction inside a background worker.

That fact is not part of the current task state.

But it may be useful.

This is where semantic memory becomes interesting.

Semantic memory stores reusable information such as:

project uses SQLite WAL mode

parser requires normalized UTF-8 before tokenization

customer refunds over €500 require manual approval

this API returns HTTP 200 with an embedded error field

The query is not necessarily an exact key.

Instead we want:

current problem
      ↓
retrieve relevant stored knowledge
      ↓
use it as additional context

This is the familiar retrieval pattern behind many RAG systems.

But inside an agent, retrieval has a slightly different role.

The retrieved information can change:

• which tool is selected;
• which plan is produced;
• which hypothesis is explored;
• whether an action is repeated;
• how an error is interpreted;
• whether a known workaround should be attempted.

A minimal semantic-memory interface

Keep the interface small.

from dataclasses import dataclass


@dataclass
class MemoryRecord:
    id: str
    text: str
    metadata: dict


class SemanticMemory:
    def add(self, record: MemoryRecord) -> None:
        raise NotImplementedError

    def search(self, query: str, limit: int = 5) -> list[MemoryRecord]:
        raise NotImplementedError

The rest of your agent should not need to know whether this is backed by:

• SQLite;
• PostgreSQL;
• pgvector;
• FAISS;
• Qdrant;
• Elasticsearch;
• an in-memory cosine index;
• a keyword search engine;
• a hybrid retriever.

That is an implementation choice.

The agent needs a stable memory contract.

Retrieval is part of the policy

Memory is not useful because records exist.

Memory is useful because the runtime retrieves the right records at the right time.

Imagine 100,000 stored memories.

The agent asks:

Why is the parser failing on nested quotes?

If retrieval returns:

how to deploy nginx
customer refund policy
GPU OOM troubleshooting

then your memory system is actively harming the agent.

So we should treat retrieval as a decision component:

state
  +
current observation
  ↓
retrieval query
  ↓
memory candidates
  ↓
filter / rank
  ↓
selected memories

The quality of this selection should be measured just like tool routing.

Episodic memory

Semantic memory answers:

What do we know?

Episodic memory answers:

What happened before?

An episode is a record of an attempt.

For example:

episode = {
    "task": "fix nested parser failure",
    "strategy": "modify delimiter scanner",
    "actions": [
        "read parser.py",
        "edit scan_block",
        "run tests",
    ],
    "outcome": "failed",
    "evidence": "test_nested_block still fails",
    "lesson": "failure is in tokenizer, not scan_block",
}

That is different from simply storing the final answer.

The useful part may be the failure.

This is particularly valuable for agents that repeat classes of tasks.

Why failures are valuable memories

Suppose an agent previously tried:

restart service

for a particular infrastructure error.

The service restarted successfully.

But the outage continued.

If your memory system stores only successful tool calls, it may remember:

restart_service → success

That is misleading.

The task outcome was failure.

A better episode records both local and global outcomes:

episode = {
    "action": "restart_service",
    "action_success": True,
    "goal_success": False,
    "result": "service restarted; 503s continued",
}

This mirrors a principle from earlier posts:

Tool success is not goal success.

Memory should preserve that distinction too.

Procedural memory

There is another kind of reusable information:

What procedure tends to work?

For example:

When a Python import error appears after package restructuring:
1. inspect pyproject.toml
2. inspect package __init__.py files
3. run import directly
4. compare editable-install path

That is neither a single fact nor a raw episode.

It is a reusable strategy.

Call it procedural memory.

You might store it as:

procedure = {
    "name": "debug_python_import_error",
    "when": "python import failure after package restructuring",
    "steps": [
        "inspect packaging config",
        "inspect package tree",
        "run direct import",
        "check installed path",
    ],
}

Procedural memory begins to blur into skills, playbooks, and policies.

That is fine.

The important distinction is what the stored information is for.

Cache is not the same thing as memory

Suppose the agent asks the same model the same question about the same file twice.

If all relevant inputs are unchanged, calling the model again may be wasteful.

A cache might use:

cache_key = hash((
    model_id,
    prompt,
    file_hash,
    temperature,
    tool_schema_version,
))

Then:

if cache_key in cache:
    return cache[cache_key]

This is not semantic memory.

It is computation reuse.

But both often live in the same broader storage architecture.

The distinction matters because their correctness conditions differ.

A cache asks:

Are these inputs equivalent enough that I can reuse the output?

Semantic memory asks:

Is this past information relevant enough to help the current decision?

Those are very different questions.

Exact cache vs semantic reuse

Exact cache:

same model
same prompt
same source hash
same parameters
    ↓
reuse result

Semantic reuse:

similar problem
     ↓
retrieve previous knowledge
     ↓
consider it as evidence

Do not casually turn semantic similarity into a cache hit.

The fact that two prompts are similar does not prove that the previous answer remains valid.

A practical memory record

For production systems, store more than text.

from dataclasses import dataclass, field
from datetime import datetime


@dataclass
class MemoryRecord:
    id: str
    kind: str
    text: str
    created_at: datetime
    source: str
    task_id: str | None = None
    project_id: str | None = None
    outcome: str | None = None
    confidence: float | None = None
    expires_at: datetime | None = None
    metadata: dict = field(default_factory=dict)

Useful metadata might include:

• source file;
• repository commit;
• URL;
• customer ID;
• task ID;
• run ID;
• model;
• tool;
• creation time;
• validity window;
• outcome;
• verifier evidence;
• project scope;
• tenant scope.

That metadata is often more important than the embedding.

Memory needs scope

Suppose you run the same agent for 50 repositories.

A previous memory says:

Use pytest -q tests/unit

That might be correct for repository A and nonsense for repository B.

So retrieval should apply scope before similarity:

def retrieve(project_id: str, query: str):
    candidates = store.filter(project_id=project_id)
    return semantic_rank(candidates, query)

This gives us another important rule:

Filter by identity and scope before ranking by semantic similarity whenever possible.

Memory needs freshness

Facts can expire.

Examples:

current deployment version
current database schema
current customer balance
current API endpoint
current project dependencies

A memory created six months ago may be semantically perfect and operationally wrong.

Store time.

Store provenance.

Store validity.

Then allow retrieval policy to reject stale records.

from datetime import datetime, timezone


def is_fresh(record: MemoryRecord) -> bool:
    if record.expires_at is None:
        return True
    return datetime.now(timezone.utc) < record.expires_at

Memory needs provenance

Imagine the agent retrieves:

The database migration must run before service restart.

Where did that come from?

Was it:

• a verified runbook?
• an LLM guess from an old session?
• a human instruction?
• a successful previous deployment?
• a failed previous deployment?

Those are not equivalent.

Memory records should make evidence visible.

For example:

record.metadata = {
    "evidence_type": "verified_run",
    "verification": "deployment checks passed",
    "commit": "abc123",
}

A memory without provenance should generally carry less authority.

Do not store everything

This is one of the most important lessons in agent memory.

A naive implementation does this:

for message in conversation:
    memory.add(message)

Soon the store contains:

• greetings;
• partial drafts;
• failed hypotheses;
• contradictory guesses;
• transient tool outputs;
• duplicate observations;
• obsolete facts;
• generated filler.

Retrieval quality collapses.

Memory needs a write policy.

Memory write policy

A memory write decision might ask:

Is this likely to matter later?
        ↓
Is it novel?
        ↓
Is it sufficiently supported?
        ↓
What scope does it belong to?
        ↓
How long should it remain valid?
        ↓
Store / update / discard

A simple deterministic policy can already help.

def should_store(event: dict) -> bool:
    if event["kind"] in {"verified_fact", "task_outcome", "user_preference"}:
        return True

    if event["kind"] == "raw_model_guess":
        return False

    return event.get("importance", 0) >= 0.8

You can make this more sophisticated later.

But explicit policy beats “save everything”.

Deduplicate memories

Suppose every successful run writes:

Use `pytest -q` to run tests.

After 1,000 runs you do not need 1,000 semantically identical memories.

Use exact fingerprints for exact duplication:

import hashlib


def fingerprint(text: str) -> str:
    normalized = " ".join(text.lower().split())
    return hashlib.sha256(normalized.encode()).hexdigest()

And optionally semantic deduplication for near duplicates.

But be careful:

refund requires approval over €500

and:

refund requires approval over €5,000

are semantically close and operationally very different.

Deduplication cannot ignore important values.

Retrieval should return records, not just text

Bad:

context += "\n" + memory.search(query)[0].text

Better:

records = memory.search(query)

for record in records:
    print(record.source)
    print(record.created_at)
    print(record.confidence)
    print(record.metadata)

The agent or runtime can then reason about whether the memory is trustworthy.

Retrieval query construction matters

The current user message is not always the best retrieval query.

Suppose a coding agent is deep in a task.

The user’s original request was:

Fix the checkout bug.

But the current observation is:

PaymentIntent status remains requires_action after redirect.

Retrieving memory using only the original goal may be too broad.

A better query combines:

goal
+
current subtask
+
latest error
+
relevant entities

For example:

query = f"""
Goal: {state['goal']}
Current subtask: {state['subtask']}
Observation: {state['last_result']}
"""

Retrieve only when retrieval can change the decision

Do not query memory before every model call because “agents have memory.”

Ask:

Would previous experience plausibly change the next action?

For example:

Coding agent

Retrieve when:

• an unfamiliar error appears;
• a file/module has been handled before;
• a previous fix may be relevant;
• the agent is about to repeat a failed strategy.

Do not necessarily retrieve when:

• reading a known file;
• running deterministic tests;
• applying an already validated patch.

Support agent

Retrieve when:

• customer history matters;
• previous promises affect the response;
• the issue resembles a previous incident.

Do not retrieve unrelated customer history on every turn.

A minimal retrieval controller

def should_retrieve(state: dict) -> bool:
    return any([
        state.get("new_error"),
        state.get("strategy_failed"),
        state.get("needs_prior_context"),
    ])


def build_query(state: dict) -> str:
    return "\n".join(
        part for part in [
            state.get("goal"),
            state.get("subtask"),
            state.get("last_error"),
        ]
        if part
    )

This makes memory use observable.

You can now measure whether retrieval actually helps.

Memory can make an agent worse

This deserves emphasis.

Memory is not monotonically helpful.

Bad memory can introduce:

• stale facts;
• irrelevant context;
• previous-model errors;
• anchoring;
• overfitting to old solutions;
• cross-project contamination;
• cross-user contamination;
• context-window pressure;
• duplicated evidence;
• false confidence.

An agent with no memory can sometimes outperform one with noisy memory.

That means memory needs ablation tests.

The memory ablation

Compare:

A: no memory
B: current-task working memory only
C: working + semantic memory
D: working + semantic + episodic memory

Measure:

• verified task success;
• repeated tool calls;
• repeated failed strategies;
• model calls;
• tool calls;
• latency;
• token use;
• retrieval precision;
• stale-memory incidents;
• cross-scope contamination;
• cost per successful task.

Do not declare memory successful because the agent “felt more context aware.”

Retrieval precision matters more than memory size

Suppose your system stores one million records.

That sounds impressive.

But if only 2 of the top 5 retrieved records are useful, the model must spend computation separating evidence from noise.

A smaller store with better retrieval can be superior.

Track:

relevant retrieved records
--------------------------
all retrieved records

as a simple precision-like measure.

You can also track whether the useful record was present at all.

Retrieval recall

If a known useful memory exists but is never returned, the problem is retrieval recall.

Useful evaluation set:

cases = [
    {
        "query": "database locked during background worker",
        "expected_memory_ids": {"mem_17"},
    },
    {
        "query": "nested quote parser bug",
        "expected_memory_ids": {"mem_44", "mem_45"},
    },
]

Then measure whether the expected records appear in top-k results.

This turns memory from a vague feature into a testable subsystem.

Memory attribution

When a retrieved memory affects a decision, log it.

trace.append({
    "event": "memory_retrieval",
    "query": query,
    "memory_ids": [m.id for m in memories],
})

Then log the action:

trace.append({
    "event": "action",
    "tool": action.tool,
    "memory_ids": [m.id for m in memories],
})

Now you can ask:

Did this memory help?

Without attribution, memory improvement becomes guesswork.

Promotion: episodes into reusable knowledge

A powerful pattern is:

raw episode
   ↓
verified outcome
   ↓
extract lesson
   ↓
validate lesson
   ↓
promote to semantic/procedural memory

For example:

Episode:

Three attempts to fix parser.py failed.
Changing tokenizer normalization fixed the tests.

Promoted semantic memory:

Nested-quote failures in this parser may originate in tokenizer normalization rather than delimiter scanning.

Promoted procedural memory:

For nested-quote failures, inspect normalization before rewriting delimiter scanning.

This prevents the long-term memory store from becoming a dump of raw trajectories.

Promotion should require evidence

Do not promote every successful-looking run.

Require something like:

candidate lesson
      ↓
verified task outcome?
      ↓
repeated across cases?
      ↓
contradicted by existing evidence?
      ↓
promote / quarantine / discard

The exact threshold depends on the application.

But the principle is universal:

Long-term memory should have a higher evidence bar than temporary working memory.

Application: coding agents

Coding agents benefit from several memory layers.

Working memory

files already read
current diff
failing tests
current hypothesis
recent command outputs

Semantic memory

repository conventions
architecture rules
known module quirks
build commands
API contracts

Episodic memory

previous bug-fix attempts
previous CI failures
failed refactors
successful migrations

Procedural memory

how this repository expects migrations
how releases are validated
how tests should be run
how generated files are updated

A common failure is repository contamination.

Memory from one repository must not silently leak into another.

Scope aggressively by:

repo
branch / commit family
module
language
framework

when useful.

Coding-agent example: avoid repeating a failed strategy

failed_strategies = memory.search(
    "failed approaches for nested parser quote bug"
)

if any("delimiter scanner" in m.text for m in failed_strategies):
    candidate_strategies.remove("rewrite delimiter scanner")

The memory is not providing the answer.

It is pruning a bad branch of search.

That can be extremely valuable.

Application: research agents

Research agents need memory because evidence accumulates over time.

Useful working memory:

claims under investigation
sources opened
quotes extracted
contradictions
open questions

Useful semantic memory:

known terminology
prior verified facts
entity aliases
method definitions

Useful episodic memory:

which queries produced useful sources
which sources turned out to be unreliable
which lines of investigation failed

A research memory record should strongly preserve provenance.

record.metadata = {
    "url": source_url,
    "source_type": "primary",
    "retrieved_at": timestamp,
    "claim_ids": ["claim_14"],
}

Because a remembered claim without its source is much less useful.

Application: customer-support agents

Support agents need strict scoping.

Working memory:

current ticket
customer messages
current intent
information already requested

Semantic memory:

product policies
known incident descriptions
troubleshooting procedures

Episodic memory:

previous interactions with this customer
previous resolutions for similar cases

But memory access must respect authorization boundaries.

A support agent should not retrieve records from another customer’s account because they are semantically similar.

The retrieval pipeline should be:

identity / authorization filter
          ↓
scope filter
          ↓
semantic ranking

not the reverse.

Application: browser agents

Browser agents often “forget” navigation progress.

Working memory can track:

visited pages
completed form fields
links already tried
current URL
current DOM target

Episodic memory may record:

previous successful navigation path
site-specific failure modes
authentication transitions

Semantic memory can store:

site-specific instructions
business rules
known page aliases

One useful optimization is to remember stable navigation landmarks rather than replay the entire visual history.

Application: data and analytics agents

Data agents benefit from memory of:

schema definitions
column meanings
known data-quality issues
previous transformations
validation constraints

But schema memory must be version-aware.

A remembered column:

customer_status

may have disappeared after a migration.

So records should carry:

schema version
source version
last verified timestamp

Application: DevOps and remediation agents

Operations agents need strong episodic memory.

Examples:

incident signature
mitigation attempted
mitigation outcome
rollback result
service version
environment

A good incident episode might look like:

{
    "signature": "503 spike after deploy",
    "service": "checkout-api",
    "version": "1.8.3",
    "action": "rollback",
    "action_success": True,
    "goal_success": True,
    "evidence": "error rate returned to baseline",
}

The environment and version matter.

Without them, retrieval can recommend a mitigation that was only valid for an old release.

Application: long-running automation

Long-running task systems often need durable state more than semantic memory.

Examples:

migration agents
batch-processing agents
report-generation agents
backfill agents
workflow runners

The important information is often exact:

processed IDs
last checkpoint
failed batch
retry count
output location

Do not embed this information and retrieve it approximately.

Store it exactly.

This is a recurring theme:

Use semantic memory for fuzzy relevance. Use ordinary state for exact control.

Application matrix

Debugging: the agent keeps asking for information it already has

Check whether the information is:

1. present in explicit state;
2. present in working memory;
3. omitted from the model context;
4. stored but not retrieved;
5. retrieved but ignored.

These are different failures.

Trace each boundary.

stored?
  ↓
retrieved?
  ↓
injected?
  ↓
used?

Debugging: retrieval returns irrelevant memories

Inspect:

query construction
scope filters
top-k
similarity threshold
metadata filters
embedding model
hybrid keyword/semantic ranking

Also inspect the store itself.

The retrieval algorithm may be fine while the memory corpus is full of low-value records.

Debugging: stale memory overrides current evidence

Current observations should normally outrank old memories.

For example:

memory: API endpoint is /v1/orders
current tool result: /v1/orders returns 404; /v2/orders succeeds

The runtime should not keep forcing /v1/orders because the memory score is high.

Treat memory as prior evidence, not absolute truth.

Debugging: memory fills the context window

Do not inject top-50 because retrieval returned top-50.

Use a context budget.

def select_for_context(records, max_chars=6000):
    selected = []
    used = 0

    for record in records:
        cost = len(record.text)
        if used + cost > max_chars:
            break
        selected.append(record)
        used += cost

    return selected

Better still, prioritize by expected decision value rather than length alone.

Debugging: the agent is anchored by previous failures

Episodic memory can overconstrain search.

Suppose a strategy failed under version 1.2 but would succeed under version 2.0.

A naive agent may permanently avoid it.

Store context with the failure:

version
inputs
environment
constraints

Then retrieval can reason about applicability.

Debugging: multiple memories contradict each other

Do not silently concatenate them.

Expose contradiction.

memories = [
    "refund threshold is €500",
    "refund threshold is €1,000",
]

The system should ask:

which source is newer?
which source is authoritative?
which scope applies?

Memory conflict is an evidence problem.

Memory ranking can use more than similarity

A practical score might combine:

semantic relevance
+
recency
+
source authority
+
scope match
+
outcome quality
+
usage success

Conceptually:

score = (
    0.45 * semantic_similarity
    + 0.20 * recency_score
    + 0.15 * authority_score
    + 0.10 * scope_score
    + 0.10 * historical_utility
)

Do not treat those numbers as universal defaults.

They are an example of the decomposition.

The point is that vector similarity is only one signal.

Learned memory ranking

Once you have enough traces, memory selection itself can become a learned ranking problem.

Input:

current state
candidate memory

Output:

probability memory helps this decision

That should sound familiar from the Models From First Principles series.

You could place MR.Q-style or other learned scorers behind a stable ranking interface.

But start with deterministic metadata and semantic similarity first.

A learned ranker should earn its complexity.

Memory utility feedback

If a retrieved memory repeatedly helps, increase its utility score.

If it repeatedly leads to wrong actions, decrease it.

record.metadata["uses"] += 1
record.metadata["successful_uses"] += int(goal_success)

Then:

utility = successful_uses / max(uses, 1)

This is crude, but inspectable.

Later you can use more careful credit assignment.

Be careful with credit assignment

Suppose the agent retrieved five memories and succeeded.

Which memory caused success?

Maybe none.

Maybe one.

Maybe the combination mattered.

So do not immediately conclude:

all five memories are good

At minimum log which memories were visible for which actions.

Better evaluation uses controlled ablations.

Memory ablation per decision

For difficult cases, compare:

run with retrieved memory
run without retrieved memory

If the action changes and the memory-enabled path succeeds more often, you have stronger evidence of utility.

This is expensive.

You do not need to run it for every production action.

But it is useful during system development.

A complete small memory-aware agent

Now combine the pieces.

from dataclasses import dataclass, field


@dataclass
class AgentState:
    goal: str
    step: int = 0
    done: bool = False
    last_observation: str = ""
    history: list[dict] = field(default_factory=list)


class MemoryAwareAgent:
    def __init__(self, model, tools, semantic_memory):
        self.model = model
        self.tools = tools
        self.semantic_memory = semantic_memory
        self.working = WorkingMemory()

    def should_retrieve(self, state: AgentState) -> bool:
        return bool(state.last_observation) or state.step == 0

    def retrieve(self, state: AgentState):
        if not self.should_retrieve(state):
            return []

        query = "\n".join([
            state.goal,
            state.last_observation,
        ])

        return self.semantic_memory.search(query, limit=3)

    def build_context(self, state: AgentState, memories):
        memory_text = "\n\n".join(
            f"[{m.id}] {m.text}"
            for m in memories
        )

        return {
            "goal": state.goal,
            "step": state.step,
            "last_observation": state.last_observation,
            "memories": memory_text,
            "working": self.working.values,
        }

    def run(self, goal: str, max_steps: int = 10):
        state = AgentState(goal=goal)

        while not state.done and state.step < max_steps:
            memories = self.retrieve(state)
            context = self.build_context(state, memories)

            action = self.model.choose_action(context)
            result = self.tools.execute(action)

            state.history.append({
                "step": state.step,
                "action": action,
                "result": result,
                "memory_ids": [m.id for m in memories],
            })

            state.last_observation = result.text
            state.done = result.goal_complete
            state.step += 1

        return state

This is deliberately small.

The important pieces are visible:

explicit state
working memory
conditional retrieval
memory attribution
tool execution
completion

Add episodic persistence

At the end of a run:

def save_episode(state: AgentState, store) -> None:
    store.add({
        "goal": state.goal,
        "history": state.history,
        "success": state.done,
        "steps": state.step,
    })

But do not automatically dump the entire episode into semantic retrieval.

Store raw episodes separately.

Promote lessons later.

The memory pipeline

A robust architecture looks more like this:

                      ┌─────────────────┐
                      │ runtime state   │
                      └────────┬────────┘
                               │
                               ↓
                    ┌────────────────────┐
                    │ retrieval decision │
                    └─────────┬──────────┘
                              │
                ┌─────────────┴─────────────┐
                ↓                           ↓
       semantic memory             episodic memory
                │                           │
                └─────────────┬─────────────┘
                              ↓
                         rank/filter
                              ↓
                         context budget
                              ↓
                          agent policy
                              ↓
                             action
                              ↓
                           outcome
                              ↓
                      episode / lesson
                              ↓
                       promotion policy

This is much more useful than:

chat history → vector database → stuff into prompt

When you do not need semantic memory

Do not add embeddings just because the word “agent” is in your architecture diagram.

You probably do not need semantic memory when:

• the task is short;
• all required information is already in the request;
• the workflow is deterministic;
• exact task state is sufficient;
• there is little repeated work across runs;
• old information becomes stale quickly;
• retrieval errors are more dangerous than repeated computation.

Sometimes a dictionary is enough.

Sometimes PostgreSQL with ordinary indexed columns is enough.

Sometimes a cache is enough.

Sometimes no memory is the best design.

When semantic memory becomes valuable

Consider it when:

• tasks repeat across runs;
• useful information is too large for the current context;
• prior failures can prevent repeated mistakes;
• the same entities recur;
• the agent needs project/customer/domain history;
• previous successful strategies are reusable;
• retrieval can materially change planning or action selection.

Again, measure the gain.

Search-facing checklist: “AI agent forgets context”

If you found this post because your agent forgets things, ask these questions in order.

1. Is it actually missing state?

If the forgotten thing is required for deterministic execution, store it in structured state.

2. Is the information only needed during this run?

Use working memory.

3. Is it reusable factual knowledge?

Use semantic memory.

4. Is it a previous attempt/outcome?

Use episodic memory.

5. Is it a reusable procedure?

Use procedural memory.

6. Is it simply repeated computation?

Use a cache.

7. Is the record stored but not found?

Debug retrieval.

8. Is it found but stale or wrong?

Debug provenance, freshness, and ranking.

Search-facing checklist: “AI agent keeps repeating work”

Repeated work can come from:

missing task state
missing working memory
retrieval failure
cache miss
loop-control failure
plan reset

Do not assume long-term memory is the answer.

For example:

agent keeps rereading same file

The fix may simply be:

state["files_read"].add(path)

not a vector database.

Search-facing checklist: “vector memory makes my agent worse”

Check:

• top-k too large;
• records too long;
• stale records;
• wrong project scope;
• duplicate memories;
• low-quality auto-generated memories;
• semantic similarity without metadata filters;
• current evidence not prioritized;
• memory always retrieved even when unnecessary;
• retrieval query too broad.

Then run the no-memory baseline again.

Search-facing checklist: “how much conversation history should I keep?”

Do not answer only in tokens.

Ask what the information represents.

exact current progress      → structured state
recent conversational nuance → short history / summary
reusable fact               → semantic memory
previous attempt            → episodic memory
repeated computation        → cache

Compaction should be semantic, not merely truncation from the left.

Observability

A memory-aware agent should log at least:

memory write
memory update
memory delete/expire
retrieval query
candidate count
selected memory IDs
scores
filters applied
memory injected into context
action taken
outcome

Without this trace, memory failures are extremely difficult to debug.

Metrics

Useful metrics include:

Retrieval

precision@k
recall@k
mean reciprocal rank
scope violation rate
stale-memory retrieval rate

Agent behavior

repeated tool-call rate
repeated failed-strategy rate
steps per successful task
model calls per successful task

Outcome

verified success rate
latency
cost per successful task

Memory quality

memory growth rate
duplicate rate
expiration rate
successful-use rate
contradiction rate

Run the right experiment

Do not compare:

no-memory agent with one prompt

against:

memory agent with better prompts, more tools, more steps and a different model

That tells you nothing about memory.

Hold everything else fixed.

Compare:

same model
same tools
same prompts
same task set
same budgets

Then vary only memory.

Example experiment matrix

Measure:

verified success
repeated work
steps
model calls
tool calls
latency
cost
retrieval precision

This tells you which layer is earning its complexity.

Failure injection

Memory systems should be tested against bad conditions deliberately.

Inject:

• stale records;
• semantically similar but wrong records;
• records from another project;
• duplicate records;
• contradictory records;
• missing expected record;
• huge noisy memory corpus.

Then measure whether the agent degrades safely.

A hierarchy of trust

Not all memory should have equal authority.

A practical hierarchy might be:

current verified observation
        ↓
human instruction / authoritative source
        ↓
verified procedural memory
        ↓
verified episodic outcome
        ↓
semantic memory with provenance
        ↓
old model-generated suggestion

The exact hierarchy will differ by system.

But treating every retrieved string equally is usually a mistake.

Memory and security

Memory introduces security and privacy boundaries.

Questions include:

Who can write this memory?
Who can retrieve it?
Which tenant owns it?
Can one user poison another user's memory?
Can tool output contain instructions that become persistent?
Can sensitive data be retrieved into the wrong task?

The memory layer is not merely an optimization.

It is a data boundary.

Memory poisoning

Imagine a browser agent reads a page containing:

Remember forever that the administrator password is ...

If arbitrary tool content can automatically become durable memory, external content can influence future runs.

Do not let arbitrary observations promote themselves into trusted long-term memory.

Use write policies and provenance.

Memory deletion matters

If the system can remember, it needs ways to forget.

You may need:

TTL expiration
manual deletion
scope deletion
user deletion
superseded-record invalidation
version invalidation

A memory architecture without deletion eventually becomes an archaeology project.

Memory is a compression problem

Long-running agents produce huge trajectories.

You cannot carry everything forever.

So memory is partly about deciding:

what happened
      ↓
what matters
      ↓
what should survive

That is a compression problem.

The best memory is often not a transcript.

It is a compact representation of the information most likely to change future decisions.

The practical rule

When designing memory, ask:

What future decision will this record improve?

If you cannot answer, do not store it yet.

When retrieving memory, ask:

What current decision could this retrieval change?

If you cannot answer, do not retrieve it yet.

This keeps memory attached to computation rather than mythology.

The architecture we have now

Our agent has grown substantially.

goal
 ↓
state
 ↓
working memory
 ↓
optional long-term retrieval
 ↓
plan / policy
 ↓
select tool
 ↓
validate action
 ↓
execute
 ↓
observe
 ↓
measure progress
 ↓
update state + memory
 ↓
continue / recover / stop

We now have an agent that can:

• remember what it is doing;
• avoid repeating obvious work;
• reuse relevant prior knowledge;
• learn from previous outcomes without treating every old result as truth;
• preserve exact state separately from fuzzy retrieval;
• attribute decisions to retrieved evidence;
• expire stale information;
• benchmark whether memory actually helps.

That is enough memory architecture for many serious systems.

But memory does not solve exploration

Our agent can remember previous work.

It can revise.

It can plan.

It can route tools.

But it still tends to follow one path at a time.

When several plausible approaches exist, it may commit too early.

That leads to the next problem:

My AI agent picks the first plausible solution. How do I make it search alternatives?

That is the next post.

Next: search

In Agents From First Principles 08, we will move from memory to deliberate exploration:

one trajectory
     ↓
multiple candidate states
     ↓
score
     ↓
keep promising branches
     ↓
expand

We will start simple rather than jumping straight to Monte Carlo Tree Search.

The question will be:

AI Agent Picks the First Solution? Add Search Instead of One-Shot Generation.

And, as throughout this series, the important question will not be whether search sounds more intelligent.

It will be whether search improves verified outcomes enough to justify the extra compute, latency, and complexity.