---
title: "LoCoMo: the long-conversation benchmark, explained"
description: "What the LoCoMo benchmark measures, the documented answer-key problem and community corrections, and how to publish comparable LoCoMo results."
canonical: https://past.dev/benchmarks/locomo
last-updated: 2026-08-31
---
# The LoCoMo benchmark

Source: https://past.dev/benchmarks/locomo

The LoCoMo benchmark evaluates long-term conversational memory over multi-session dialogues. Questions cover single-hop recall, multi-hop reasoning, open-domain knowledge and temporal reasoning. Its original answer keys contain documented errors. A reported score should therefore state the key version, retrieval limits and judge configuration.

## What LoCoMo contains

LoCoMo, from Snap Research at [github.com/snap-research/locomo](https://github.com/snap-research/locomo) with the paper at [arxiv.org/abs/2402.17753](https://arxiv.org/abs/2402.17753), generates very long multi-session conversations between personas and asks questions that require remembering across sessions. The question mix covers single-hop lookups, multi-hop combinations, open-domain questions and temporal reasoning about when things happened.

## The answer-key problem

The community has documented errors in some original LoCoMo answer keys. Different correction sets produce different scores. We use community-corrected keys and identify them on the [benchmarks page](/benchmarks). Every published LoCoMo result should name its key version.

## Publishing a comparable LoCoMo result

- Name the answer-key version and any exclusions.
- Publish retrieval limits, token budget and the judge configuration.
- Report retrieval sufficiency and answer accuracy separately to identify whether retrieval or the answer model failed.
- Report cost and latency from the same run as the accuracy number.
- Publish the harness so another team can rerun the evaluation.

## What LoCoMo does not test

LoCoMo uses conversations between two personas. It does not test backdated ingestion from mixed sources, entity resolution across email and tickets or point-in-time queries over operational data. These functions are described in [what is agent memory](/what-is-agent-memory) and [point-in-time recall](/glossary/point-in-time-recall). Pair LoCoMo with suites that test updates and abstention, such as [LongMemEval](/benchmarks/longmemeval). The [benchmarks page](/benchmarks) documents our method. The API [quickstart](/docs/memory-api/quickstart) shows how to test your own data.

## Frequently asked questions

### Why do LoCoMo scores differ so much between vendors?

Key versions, retrieval limits, token budgets and judges all differ, and each moves the number. Comparisons are valid only across identical configurations, which is why the harness matters more than the headline.

## Related

- [LongMemEval, explained](https://past.dev/benchmarks/longmemeval)
- [How to benchmark agent memory](https://past.dev/benchmarks/methodology)
- [MemoryBench, explained](https://past.dev/benchmarks/memorybench)
- [What is agent memory?](https://past.dev/what-is-agent-memory)
- [Quickstart](https://past.dev/docs/memory-api/quickstart)