--- title: "Artificial Scientist Swarms: boredom and memory" description: "A research swarm measures its own boredom and changes direction. Its agents share hypotheses, results and lessons in past.dev." canonical: https://past.dev/blog/projects/artificial-scientist-swarms last-updated: 2026-11-18 --- # Artificial Scientist Swarms: boredom and memory Projects, November 17, 2026. The past.dev team. Team caffeinated built Artificial Scientist Swarms at the Iterate AI Research Hackathon. The team is Ariel Anders, Jialin Feng, Neeraj Shah and Masha Baidachna. A swarm of AI agents runs research experiments and measures its own boredom. When its ideas stop giving better results, the swarm changes direction. The agents share their hypotheses, results and lessons in one memory on past.dev. Demo video: [A demo of Artificial Scientist Swarms by the team](https://www.youtube.com/watch?v=qlji8ruy6MU). The image shows three robot judges and the question "Who will win the Ig Nobel Prize?". - Built by: Ariel Anders ([GitHub](https://github.com/arii)), Jialin Feng ([GitHub](https://github.com/jfeng012)), Neeraj Shah, Masha Baidachna ([GitHub](https://github.com/mashathepotato), [LinkedIn](https://www.linkedin.com/in/masha-baidachna)) - Event: [Iterate AI Research Hackathon](https://iterate.inc/ai-research), Wefunder, San Francisco, November 14 and 15, 2026 - Live lab replay: https://arii.github.io/caffeinated - Code on GitHub: https://github.com/arii/caffeinated ## The project Artificial Scientist Swarms is a framework for automated research. In each round, four AI agents, the workers, each propose one idea for a research task. A sandbox tests each idea, and the swarm keeps the best result. A conductor gives each worker its role, and a critic checks each new best score. The swarm writes each test to the ledger, a file with each attempt and its score. The team started from a hypothesis: swarms of AI scientists get stuck because they are bored. They keep trying small variants of one idea after the gains stop. A person reacts in a different way: in psychology, boredom is a signal to look for something new ([sources](https://github.com/arii/caffeinated/blob/main/boredom_meter/README.md#references)). The team wanted a swarm that notices its boredom and acts on it. So the team turned the signal into a number, the Boredom Meter. The meter is `B = 0.45 * (1 - novelty) + 0.45 * stagnation + 0.10 * overload`, from 0 to 1. Novelty is low when recent attempts are similar to earlier attempts. Stagnation grows with the number of rounds since the best score last improved by more than the noise level. Overload grows when attempts fail or repeat earlier attempts. Any agent can calculate B from the ledger, without a call to an LLM. | Mode | When | What the four workers do | |---|---|---| | Exploit | B is low, or the best score just improved | 3 refine the best idea, 1 tries a different idea | | Boredom snap | B is 0.70 or more | 1 refines, 3 try different ideas. A filter rejects ideas that are too close to recent attempts. | | Paradigm breakout | B is 0.85 or more, and the best score did not change for a set number of rounds | All 4 try ideas with a new main mechanism | The workers propose ideas at the same time. Each worker must know what the other workers tried and what they test now. When a worker is bored, it must also know which directions failed, so that it can find a new direction. Later runs of the same task need the lessons of earlier runs. The team tested the swarm on an Ig Nobel task: find research that is funny at first and then gives a real insight. The agents find papers and public data sets with the Exa search API, and the swarm tests each hypothesis once on real data. A judge model scored 60 ideas from 4 runs. The idea with the [best score](https://github.com/arii/caffeinated/blob/main/data/igjudge/ignobel-real-runs/results-zip.md) got 76.5 of 100: "The Brazil Nut Canary: A Snack-Sized Early-Warning System for Granular Avalanches". The project placed second in the past.dev challenge of the hackathon. It was also a finalist in the track "Build your own autoresearch framework". ## How the memory works The swarm calls the past.dev REST API from one Python file, [memory.py](https://github.com/arii/caffeinated/blob/main/memory.py). Each record has the identity of its author, for example `worker-2` or `critic`. The ledger stays outside past.dev, and only the ledger gives the official scores. The Boredom Meter reads only the ledger, so memory cannot change B. | Agent | Writes to past.dev | Reads from past.dev | |---|---|---| | Conductor | The role of each worker in each round | Nothing in the default mode | | Scouts (workers in the scout role) | A lead: one source from Exa and the idea that it suggests | Lessons and results about directions that the swarm already tried | | Workers 1 to 4 | Each idea, the result of each test, and the status of each hypothesis | Before each new idea: what was tried, the open leads, who tests what in this round, and the status of each hypothesis | | Critic | A verdict on each new best score, and a lesson | Nothing | ### Each run reads with its own identity For each run, memory.py creates one reader identity with three traits: `arm`, `run` and `task`. The trait `arm` names the setting of the experiment, for example the swarm with or without the Boredom Meter. Each agent writes with its own identity, and each recall of the run uses the reader identity. memory.py also creates two audiences for each run with `PUT /api/v1/audiences/{slug}`. An audience is a group of identities, defined by a rule on their traits. | Audience | Who can read it | Records | |---|---|---| | `run-{arm}-{run}` | Identities with the same setting and run | Ideas, leads, roles and kept results of this run | | `lab-{arm}-{task}` | Identities with the same setting and task | Hypotheses, lessons, verdicts and failed results, for later runs | A recall with the reader identity of a run returns the records of this run. It also returns the `lab` records of earlier runs with the same setting and task. It does not return records of another setting or another task. These are the calls for one run, simplified from memory.py: ```python reader = f"reader-{arm}-{run}" api("PUT", f"/identities/{reader}", {"traits": {"arm": arm, "run": run, "task": task}}) api("POST", "/ingest", { "id": record_id, # hyp-B-... for a hypothesis "content": content, "identity": author, # worker-2, critic, ... "audience": audience, # run-... or lab-... }) api("POST", "/recall", {"query": query, "identity": reader, "limit": 5}) ``` ### Each idea has one record with its current status Each idea has one hypothesis record, and the id of this record never changes. When a worker starts to test the idea, the swarm writes the record with the status "being tested". When the test ends, the swarm writes a record with the same id and the result, for example confirmed, refuted or crashed. The new version replaces the old one. A recall then returns one current status for each idea. ### A rollback removes the records of the cancelled rounds The swarm saves a checkpoint after each round. If a run starts again from a checkpoint, the later rounds do not count. memory.py then deletes their records with `POST /api/v1/data-points/delete`, so a later recall does not return them. ## What memory changed The team ran two tests of the memory during the hackathon. The results are in [docs/past-dev.md](https://github.com/arii/caffeinated/blob/main/docs/past-dev.md) and [experiments/memory-comparison](https://github.com/arii/caffeinated/blob/main/experiments/memory-comparison/README.md). The team did not measure whether memory changed the ideas or the scores of the swarm. ### Each recall returned records from other workers In a short run with the past.dev API, all 69 recalls returned at least one record from another worker. The team expected this result, because the four workers of a run share one audience. For example, `worker-2` sent the query "current status of hypotheses already tested". The recall returned 5 hypothesis records from worker-1, worker-3 and worker-4. ### past.dev put the right record first more often than a word search The test uses all 419 memory records from 4 real swarm runs. The team picked 40 of these records and wrote one question for each record. Each question uses other words than its record. For example, the question "Are weightier automobiles less fuel-efficient in town?" looks for the record "Do heavier cars get fewer city miles per gallon?". The team sent each question to past.dev recall and to TF-IDF, a local search that matches words. | Search | Right record first | Right record in the first 8 results | |---|---|---| | past.dev recall (40 questions) | 26 of 40 | 38 of 40 | | Local TF-IDF search (40 questions) | 12 of 40 | 33 of 40 | The retrieval test of the team on 419 real records. One square is one question. past.dev was slower: its median recall took 562 ms, against 15 ms for the local search. The team lists the limits of this test. The team ran the test once, with 40 questions. Because the questions use other words than their records, the test favors a semantic search over a word search. The team did not test a local semantic search. ## Try Artificial Scientist Swarms 1. Open the [live lab replay](https://arii.github.io/caffeinated) and select Play. 2. Select a scientist to see its work. 3. Read [memory.py](https://github.com/arii/caffeinated/blob/main/memory.py). It contains each call to past.dev. To build agents that share one memory, start with the [quickstart of the Memory API](/docs/memory-api/quickstart). Thank you to Ariel, Jialin, Neeraj and Masha for building with past.dev at the hackathon.