--- title: "Memory observability for AI agents" description: "Memory observability answers why an agent retrieved what it did: what to log at ingestion and recall, how to trace a wrong answer, and what to alert on." canonical: https://past.dev/guides/memory-observability last-updated: 2026-10-17 --- # Memory observability for AI agents Source: https://past.dev/guides/memory-observability Memory observability is the ability to explain what an agent's memory did: why a fact was retrieved, why another was missed, and what the store believed at the time. It requires logging two units, the ingestion record and the recall response, each with dates, sources and status. With those logs, a wrong answer traces to a wrong source, a stale fact, a failed join or a retrieval miss in minutes; without them, memory bugs surface as model bugs and get prompt-engineered instead of fixed. ## Memory bugs masquerade as model bugs When an agent answers wrongly, the first suspect is usually the model or the prompt. In systems with a memory layer, four memory failures produce the same visible symptom, and none of them is fixable in the prompt. - **A wrong or stale source.** The store faithfully returned a fact whose source was wrong, or whose value was later changed elsewhere. See [memory staleness](/glossary/memory-staleness). - **A failed join.** The evidence exists under a second identity the resolver never linked; the retrieval was correct over a broken entity. See [entity resolution](/guides/entity-resolution). - **A retrieval miss.** The fact exists on the right entity and ranked below the cutoff; [recall precision](/glossary/recall-precision) and ranking share the blame. - **A silent conflict.** Two sources disagreed, one was picked without a record. See [contradictory facts](/guides/contradictory-facts). Distinguishing the four from the outside is guesswork. Distinguishing them from logs is a lookup, which is the case for building the logs first. ## The two loggable units Memory observability needs exactly two records kept per interaction, both of which the memory layer already produces. 1. **The ingestion record.** For each write: the source identifier, the [event time](/glossary/event-time) supplied, the ingestion time, and the final `status` of the send. Polling `GET /api/v1/ingest/{ingestionId}` until `status` is `completed` belongs in the pipeline anyway; logging that status closes the loop, because a query answered before ingestion completes reads the store as it was. 2. **The recall response.** For each read the agent acted on: the query and the returned documents with their dates and source excerpts. This is the knowledge state behind the action, and storing it turns every wrong answer into a diffable artifact. These two units are also the backbone of the [audit trail](/guides/ai-audit-trail); observability and audit are the same records consulted at different speeds. ## Tracing a wrong answer 1. Pull the recall log for the interaction. If the returned documents did not support the answer, inspect the application's sufficiency and answer-generation policy. 2. If the evidence contains the wrong fact, follow its source link. A wrong source is a data problem; a right source with an outdated value is a missing supersession, checked against [facts that change over time](/guides/facts-that-change-over-time). 3. If the evidence misses a fact you know exists, query the entity directly. Present on another identity means a resolution failure; present but unranked means a retrieval miss; absent entirely means ingestion never extracted it. 4. Reproduce with the ingestion log: the event times and ingestion statuses say what the store could have known at that moment. ## What to watch continuously A handful of rates, computed from your own recall logs, catch degradation before users do. Track empty result rate, returned document counts, answer-layer abstentions, request failures, and ingestion lag. Recalls issued before ingestion completes can read the state before an update. Thresholds are yours to set; the point is that each rate is computable from public responses and logs you already keep. ## What past.dev gives observability past.dev makes both loggable units explicit: ingestion returns an identifier whose `status` is pollable, and every recall response carries ranked documents with ranks, dates, and source excerpts. Store that response together with the application's sufficiency decision and resulting action. [Recall](/docs/memory-api/quickstart) behavior is part of what the [benchmarks](/benchmarks) evaluate. ## Frequently asked questions ### Why did my AI agent retrieve the wrong memory? One of four causes: the source itself was wrong or stale, the fact sat on an unlinked duplicate identity, ranking placed the right fact below the cutoff, or two sources conflicted and one was chosen silently. The recall log with documents, dates and source excerpts tells you which in one lookup. ### What should I log from an AI agent's memory? Two units: each ingestion with its source id, occurrence time and final status, and each recall response the agent acted on, including its ranked documents, dates, and source excerpts. Store the application's sufficiency decision beside them. ### How is memory observability different from LLM observability? LLM observability watches the model: tokens, latency, traces of calls. Memory observability watches the knowledge: what was stored, what was retrieved, with what support. A wrong answer can come from either layer, and separating them is the point of logging both. ## Related - [Audit trail for AI agents](https://past.dev/guides/ai-audit-trail) - [Contradictory facts](https://past.dev/guides/contradictory-facts) - [Testing entity resolution](https://past.dev/guides/testing-entity-resolution) - [Evidence status](https://past.dev/glossary/evidence-status) - [How recall is evaluated](https://past.dev/benchmarks/methodology)