---
title: "How memory accuracy degrades with scale"
description: "Memory accuracy measured at 100K, 500K, 1M and 10M tokens: the published splits, which abilities hold at scale, and where systems actually lose accuracy."
canonical: https://past.dev/blog/memory-accuracy-at-scale
date: 2026-09-01
category: Research
authors: The past.dev team
---
# How memory accuracy degrades with scale

Memory accuracy is usually reported at one corpus size, and the number tells you little about the size you will actually run at. We publish the full curve. On BEAM, the current run scores 92.08% at 100K tokens, 89.63% at 500K, 90.65% at 1M and 85.03% at 10M, each measured on the complete split. This post reports where those points come from and, more usefully, which abilities lose accuracy as history grows and which do not.

## The published splits

Each scale is a complete split rather than a sampled slice, and each row lists the question and conversation counts behind it, because a score is only checkable when its denominator is visible. The previous run, of 26 August 2026, stays beside the current one so the change is visible. The current figures at all four sizes come from one scoring run, of 29 September 2026.

| Scale | Current run | 26 August 2026 | Questions | Conversations |
|---|---|---|---|---|
| 100K tokens | 92.08% | 91.50% | 400 | 20 |
| 500K tokens | 89.63% | 87.97% | 700 | 35 |
| 1M tokens | 90.65% | 86.18% | 700 | 35 |
| 10M tokens | 85.03% | 82.61% | 200 | 10 |

The [benchmarks page](/benchmarks) carries the current figures with the per-ability breakdown, alongside the published results other systems report for themselves. The [BEAM leaderboard](/benchmarks/beam) ranks every published result at each size. mem0, for comparison, reports 64.1% at 1M and 48.6% at 10M in [its own benchmark writeup](https://mem0.ai/blog/ai-memory-benchmarks-in-2026).

## Where memory accuracy is lost

The overall curve drops 7.05 points across a 100x increase in history, against 8.89 in the previous run. The drop is concentrated in specific abilities. Averages hide this, so here is the profile. The ability figures below come from the current run at every size.

**What holds at scale.** Event ordering scores 99.81% or above at every size. Preference following stays at 98.87% or above, and summarization at 97.25% or above. Abstention, the ability to decline when the stored evidence does not establish an answer, is 98.75% at 100K, 92.14% at 500K, 97.14% at 1M and 100% at 10M. These abilities depend on the temporal structure of storage rather than on search over a larger haystack, which is why scale barely touches them.

**What degrades.** Multi-session reasoning, combining evidence spread across separate conversations, holds between 68.95% and 72.82% from 100K to 1M, then falls to 28.17% at 10M, the steepest decline in the table. Information extraction falls from 95% at 100K to 86.32% at 500K, 83.98% at 1M and 68.75% at 10M. Contradiction resolution scores between 73.13% and 78.57% at every scale, from 78.13% at 100K to 73.13% at 10M. These three are the frontier: they are hard at every size and harder with more history.

**What changed since the previous run.** Knowledge update, answering with the current value after a fact changed, fell to 62.14% at 1M in the run of 26 August. The current run scores 89.38% at 100K, 86.07% at 500K, 85.71% at 1M and 92.50% at 10M. Multi-session reasoning at 10M moved the other way, from 43% to 28.17%.

One caveat we would want stated if someone else published this table: the 10M split has 20 questions per ability category, against 40 at 100K and 70 at 500K and 1M. A category score at n=20 is a coarser measurement, and single-category swings at 10M should be read with that in mind.

## What this means if you are evaluating memory

Ask any memory vendor, including us, three things. First, the full curve rather than one point: a system tuned for a demo corpus can score well at 100K and collapse later, and the [500K and 1M splits](/benchmarks) are where degradation becomes visible. Second, complete splits with visible denominators, since sampled slices and hidden n make scores unfalsifiable. Third, the per-ability profile, because your workload is not an average: an agent that mostly needs event ordering has different risk than one that lives on knowledge updates.

The hard abilities are the ones that depend on [tracking facts that change over time](/guides/facts-that-change-over-time). Knowledge updates need [supersession](/glossary/fact-supersession), which is where the current run gained most, and multi-session reasoning needs [entity resolution](/guides/entity-resolution) to connect the same people across conversations. Systems that treat memory as similarity search over chunks degrade fastest on precisely these questions.

The full methodology, judges, and configurations are documented on the methodology page. [How to reproduce this →](/benchmarks/methodology)