---
title: "What is agent memory?"
description: "Agent memory is what an AI agent knows across sessions, and the mechanism that puts the right part of it back in the context window. The problems it solves, the vocabulary, the existing approaches and where each one stops."
canonical: https://past.dev/what-is-agent-memory
last-updated: 2026-08-26
---
# What is agent memory?

Source: https://past.dev/what-is-agent-memory

Agent memory is the part of an agent system that decides what the model knows about the past when it runs. A context window holds one conversation; memory holds what was learned across all of them and puts the relevant part back into the window at the right moment. In practice it covers four jobs: holding the working state of the current task, carrying facts across sessions, resolving who and what is being referred to, and knowing when something that was true has stopped being true.

## What problems agent memory solves

1. **Continuity.** The user said something three sessions ago that changes the answer now. Without memory, every session starts from zero and the user repeats themselves.
2. **Working state.** A long task accumulates decisions, and the beginning of a long run falls out of the window before the end of it arrives. Memory is where that state lives when the window cannot hold it.
3. **Identity.** The same person, account or project appears under different names in different sources. Something has to decide that they are the same subject, or the history stays split.
4. **Currency.** Facts change. A memory that returns every statement ever made about a subject has moved the problem to the model instead of solving it.

## Agent memory vs a context window

The context window is the model's working set for one call. It is bounded, it is charged for on every call, and it is empty again next time unless something refills it. Memory is the store outside it plus the policy that decides what to put in.

Larger windows change the economics but not the question. A million tokens of history sent on every call is expensive and, past a point, counterproductive: relevant evidence competes with everything else in the prompt. The decision of what to include is [context engineering](/context-engineering), and memory is what makes that decision answerable.

## The vocabulary: working, episodic, semantic, procedural

The field borrows its terms from cognitive psychology. They are useful as labels for four different storage problems, not as a claim about how models work.

| Term | What it holds | Typical implementation |
| --- | --- | --- |
| Working, or short-term | The current task: recent turns, intermediate results, the plan | The context window itself, plus a scratchpad the agent rewrites |
| Episodic | What happened, and when: past conversations and events | A log of messages or events, retrieved by similarity or by time |
| Semantic | Facts that hold independently of when they were said | Extracted statements in a store, a knowledge graph, or a profile document |
| Procedural | How to do things: instructions, tools, learned preferences | The system prompt, tool definitions, and files the agent edits |

Most production systems mix all four and call the mixture memory. When comparing two products, ask which of the four each one actually implements, because the word covers very different amounts of work.

## Agent memory vs RAG

RAG retrieves from a corpus you curated to answer a question. Memory accumulates from the agent's own history and has to maintain it: merge duplicates, supersede outdated facts, and decide what is worth keeping.

The mechanics overlap, both retrieve and both often use embeddings, so the honest distinction is not the retriever but the write path. RAG has no write path to speak of, because the corpus already exists. Memory is mostly write path.

## How existing approaches work, and where each one stops

Five designs, in rough order of how much they take on. Each is a reasonable answer to a different amount of the problem.

### Buffers and summaries

Keep the last N turns, and summarise what falls off the end. Cheap, no infrastructure, and the default in most frameworks. It stops at the first thing a summary loses: a summary is lossy in a direction nobody chose, and once a detail is gone it cannot be recovered from the summary.

### Vector recall over past messages

Embed every message and retrieve the similar ones at question time. Scales further and needs no extraction step. It stops at contradiction: two messages that disagree are both similar to the question, and nothing in the index says which one holds now.

### Self-editing memory blocks

The agent owns its memory and edits it with tools. [Letta](https://docs.letta.com), the project that came out of MemGPT, is the clearest implementation: memory blocks sit in the context window and the agent rewrites them, while archival memory sits outside and is searched on demand. It stops where any self-editing system does: correctness depends on the agent choosing to write, and on it choosing well.

### Extracted facts in a store

Run extraction over conversations, store the resulting statements, retrieve them per user or per session. [mem0](https://github.com/mem0ai/mem0) is the widely used open-source implementation of this shape. It gets a compact, readable memory, at the cost of a model in the write path and the extractor's judgement about what mattered.

### Temporal knowledge graphs

Extract entities and relationships, and give every fact a validity window so a superseded fact is marked invalid rather than deleted. [Graphiti](https://github.com/getzep/graphiti), the open-source engine behind Zep, is built this way, and so is past.dev. It is the most work per document by a wide margin, and it is the only shape that answers questions about what changed. [Knowledge graphs for LLM applications](/knowledge-graph-for-llm) covers the trade in detail.

## What makes memory hard

- **Identity.** Two names, one person. Resolution has to happen at write time and keep happening as the graph grows, because the evidence that two names are one subject often arrives after both.
- **Contradiction.** The system must be able to represent that something was true and is not any more, which means storing time as a property of the fact rather than as metadata on the document.
- **Forgetting.** A memory that never drops anything grows without bound and retrieves worse every month. Deciding what to keep is a product decision as much as a technical one.
- **Abstention.** A memory that always returns something teaches the model that context is always sufficient. Returning nothing has to be an available answer.
- **Cost.** Extraction at write time and retrieval at read time both cost money per call, which is why per-call cost belongs in the response rather than in a monthly invoice.

## How to tell if your agent needs memory

- Users repeat context they already gave you in a previous session.
- The agent answers with something that was true last quarter.
- The same subject appears under different names and the agent treats them as strangers.
- Your prompt has grown a section that pastes history in, and it keeps growing.
- You cannot answer the question, where did that answer come from.

If none of those is true, a retrieval pipeline is enough, and [choosing a vector database](/vector-database-for-rag) is the whole decision.

## Frequently asked questions

### What is agent memory?

The part of an agent system that decides what the model knows about the past when it runs: a store of what was learned across sessions, plus the policy that puts the relevant part back into the context window.

### What is the difference between agent memory and a context window?

The context window is the model's working set for one call, bounded and paid for every time. Memory is what persists outside it and what refills it. The window is capacity; memory is selection.

### Is RAG the same as agent memory?

No. RAG retrieves from a corpus that already exists. Memory accumulates from the agent's own history and has to maintain that history: merging duplicates, superseding outdated facts, resolving identity. The retrieval half looks similar; the write half is the difference.

### Do I still need memory with a one million token context window?

For anything long-running, yes. Sending the whole history on every call is expensive and gets less accurate as the window fills with material that competes for attention. A larger window raises the ceiling on what you can include; it does not decide what should be included.

### What is the difference between short-term and long-term memory in an agent?

Short-term, or working, memory is the current task's state, usually the window itself plus a scratchpad. Long-term memory is what survives the session: episodic records of what happened, semantic facts about the world, and procedural instructions about how to act.

### How do I store agent memory?

Four common options: keep raw messages and summarise, embed messages and retrieve by similarity, extract facts into a store, or build a temporal graph where each fact carries a validity window. They differ mainly in how much work happens at write time, and in whether contradiction can be represented at all.

## Related

- [Context engineering for AI agents](https://past.dev/context-engineering)
- [Vector database vs graph database vs memory](https://past.dev/vector-database-vs-memory)
- [Knowledge graphs for LLM applications](https://past.dev/knowledge-graph-for-llm)
- [Memory API overview](https://past.dev/docs/memory-api/overview)
- [How we measure memory](https://past.dev/benchmarks)