--- title: "Memory observability for AI agents" description: "Memory observability answers why an agent retrieved what it did: what to log at ingestion and recall, how to trace a wrong answer, and what to alert on." canonical: https://past.dev/guides/memory-observability last-updated: 2026-09-02 --- # Memory observability for AI agents Source: https://past.dev/guides/memory-observability Memory observability is the ability to explain what an agent's memory did: why a fact was retrieved, why another was missed, and what the store believed at the time. It requires logging two units, the ingestion record and the recall response, each with dates, sources and status. With those logs, a wrong answer traces to a wrong source, a stale fact, a failed join or a retrieval miss in minutes; without them, memory bugs surface as model bugs and get prompt-engineered instead of fixed. ## Memory bugs masquerade as model bugs When an agent answers wrongly, the first suspect is usually the model or the prompt. In systems with a memory layer, four memory failures produce the same visible symptom, and none of them is fixable in the prompt. - **A wrong or stale source.** The store faithfully returned a fact whose source was wrong, or whose value was later changed elsewhere. See [memory staleness](/glossary/memory-staleness). - **A failed join.** The evidence exists under a second identity the resolver never linked; the retrieval was correct over a broken entity. See [entity resolution](/guides/entity-resolution). - **A retrieval miss.** The fact exists on the right entity and ranked below the cutoff; [recall precision](/glossary/recall-precision) and ranking share the blame. - **A silent conflict.** Two sources disagreed, one was picked without a record. See [contradictory facts](/guides/contradictory-facts). Distinguishing the four from the outside is guesswork. Distinguishing them from logs is a lookup, which is the case for building the logs first. ## The two loggable units Memory observability needs exactly two records kept per interaction, both of which the memory layer already produces. 1. **The ingestion record.** For each write: the source identifier, the [event time](/glossary/event-time) supplied, the ingestion time, and the settlement outcome. Polling `GET /api/v1/ingest/{ingestionId}` until `settled` is true belongs in the pipeline anyway; logging the settlement closes the loop, because a query answered before settlement reads the store as it was. 2. **The recall response.** For each read the agent acted on: the query, the returned evidence with its dates and sources, and the [evidence status](/glossary/evidence-status). This is the knowledge state behind the action, and storing it turns every wrong answer into a diffable artifact. These two units are also the backbone of the [audit trail](/guides/ai-audit-trail); observability and audit are the same records consulted at different speeds. ## Tracing a wrong answer 1. Pull the recall log for the interaction. If the status was NoKnownSupport or UnknownBecauseDegraded and the agent answered anyway, the bug is in the application's status handling, and the memory layer is exonerated. 2. If the evidence contains the wrong fact, follow its source link. A wrong source is a data problem; a right source with an outdated value is a missing supersession, checked against [facts that change over time](/guides/facts-that-change-over-time). 3. If the evidence misses a fact you know exists, query the entity directly. Present on another identity means a resolution failure; present but unranked means a retrieval miss; absent entirely means ingestion never extracted it. 4. Reproduce with the ingestion log: the event times and settlement outcomes say what the store could have known at that moment. ## What to watch continuously A handful of rates, computed from your own recall logs, catch degradation before users do. Track the share of recalls returning each status value: a rising share of insufficient-evidence answers means ingestion is falling behind the questions, and a rising share of conflicts means sources are drifting apart. Track settlement lag from the ingestion records: recalls racing ahead of settlement read yesterday's store. Track new-entity rate: a sudden surge often means the resolver stopped joining and is minting duplicates. Thresholds are yours to set; the point is that each rate is computable from logs you already keep. ## What past.dev gives observability past.dev makes both loggable units explicit: ingestion returns an identifier whose settlement is pollable, and every recall response carries ranked evidence with dates, sources and one of four status values, so storing the response is storing the explanation. [Recall](/docs/memory-api/quickstart) behavior on unanswerable and conflicting questions is part of what the [benchmarks](/benchmarks) evaluate. ## Frequently asked questions ### Why did my AI agent retrieve the wrong memory? One of four causes: the source itself was wrong or stale, the fact sat on an unlinked duplicate identity, ranking placed the right fact below the cutoff, or two sources conflicted and one was chosen silently. The recall log with evidence, dates and status tells you which in one lookup. ### What should I log from an AI agent's memory? Two units: each ingestion with its source id, event time and settlement outcome, and each recall response the agent acted on, with its evidence, dates, sources and status value. Together they explain any answer after the fact. ### How is memory observability different from LLM observability? LLM observability watches the model: tokens, latency, traces of calls. Memory observability watches the knowledge: what was stored, what was retrieved, with what support. A wrong answer can come from either layer, and separating them is the point of logging both. ## Related - [Audit trail for AI agents](https://past.dev/guides/ai-audit-trail) - [Contradictory facts](https://past.dev/guides/contradictory-facts) - [Testing entity resolution](https://past.dev/guides/testing-entity-resolution) - [Evidence status](https://past.dev/glossary/evidence-status) - [How recall is evaluated](https://past.dev/benchmarks/methodology)