--- title: "MOM: long-term memory for the physical world" description: "MOM answers questions such as 'Where are my keys?' from the video of smart glasses. Its five agents share one memory on past.dev, and each record has a time." canonical: https://past.dev/blog/projects/mom last-updated: 2026-11-18 --- # MOM: long-term memory for the physical world Projects, November 17, 2026. The past.dev team. Jaspal Singh built MOM at the Iterate AI Research Hackathon. MOM keeps a memory of the video from a pair of smart glasses, and it answers questions out loud, for example "Where are my keys?". Its five agents write to one memory on past.dev with four identities, and each record has the time of its video. Demo video: [A demo of MOM by Jaspal Singh](https://www.youtube.com/watch?v=5mYZRG-iHpY). The image shows the stage page during a question about soy milk. - Built by: Jaspal Singh ([GitHub](https://github.com/jsaluja), [LinkedIn](https://www.linkedin.com/in/jaspalsaluja/)) - Event: [Iterate AI Research Hackathon](https://iterate.inc/ai-research), Wefunder, San Francisco, November 14 and 15, 2026 - Watch the demo: https://www.youtube.com/watch?v=5mYZRG-iHpY ## The project MOM is a voice assistant for smart glasses. You wear the glasses through the day with the camera on, and MOM records the objects in the video. Later, you ask out loud, for example "Hey Mom, any soy milk left?". MOM answers in a few words with what, where and when, and it shows a photo from the video on the phone. The glasses send video to an iPhone app. The app cuts the video into windows of about 9 seconds, and it sends each window to Modal, a cloud service that runs code. Five agents process the video and the questions. All of them run on the model Qwen3.5-397B through the Tinker API. If Tinker is unavailable, a fallback model on Modal takes over. | Agent | Job | |---|---| | Observer | Records each object in a window of video: what it is, where it is, its state and its label | | Comparator | Proposes what changed since the last state in memory | | Verifier | Checks each proposed change in the video without seeing the claim, and rejects false changes | | Assistant | Answers a question from memory and the live view, in 25 words at most | | Researcher | Searches the web with the Exa search API, only when a question needs it | The agents work at different times. The Observer and the Comparator write while you wear the glasses. The Assistant reads later, when you ask a question. The Comparator also needs the last state of each object, so that it can see what changed. For this reason, the agents share one memory, and each record keeps the time of its video. ## How the memory works MOM calls the past.dev REST API from one Python client. It uses four endpoints: `PUT /api/v1/audiences/{slug}`, `POST /api/v1/ingest`, `POST /api/v1/recall` and `DELETE /api/v1/data-points/{id}`. All the agents write to one project. Each record holds one item: a sighting, a change, a web answer, a watch item or a correction. When you ask about an object, the Assistant writes a watch item for it. | Agent | Identity | Writes to past.dev | Reads from past.dev | |---|---|---|---| | Observer and Comparator | `mom-observer` | Each sighting of an object, and each sign with readable text | The last state of every object, and the watchlist of the Assistant | | Verifier | `mom-verifier` | Each confirmed change, with a frame from before and a frame from after | Nothing: it gets the last state from the Observer | | Researcher | `mom-researcher` | Each web answer, with the Exa source URLs | Nothing: the Assistant passes it the question | | Assistant | `mom-assistant` | The watchlist (what you ask about), your spoken corrections, and sightings from the live view when you ask about something in front of you | Every record above, by relevance to the question | The watchlist of the Assistant also goes back to the Observer. At each window, the Observer reads what you asked about, so it watches these objects more closely. ### Each record names its writer Each write sends the `identity` of its agent. The app also puts the name of the writer in `metadata.by`. For each answer, the app counts the past.dev records in the prompt, in total and for each writer. The ask endpoint of MOM returns both counts. The stage page, the screen of the demo, shows the total under the answer. ### Two audiences for each space On past.dev, an audience is a list of identities that can read a record. A space is one place, for example `home`. For each space, the app creates two fixed audiences with `PUT /api/v1/audiences/{slug}`: - `mom-{space}` lists the four identities. Sightings, signs, changes and web answers go to this audience. - `mom-{space}-asks` lists only `mom-observer` and `mom-assistant`. The watchlist and the corrections go to this audience. The Researcher sends queries to the public web. For this reason, it must not read what you asked about. Because `mom-researcher` is not in the second audience, a recall as the Researcher does not return the watchlist or the corrections. The app also keeps a result only if its `metadata.space` is the current space. ### An endpoint tests who can read the watchlist `GET /api/boundary?space=home` is an endpoint of MOM on Modal. It sends the watchlist recall of the Observer four times, once with each identity. Then it returns the watch items in each of the four results. If the two audiences work as designed, only `mom-observer` and `mom-assistant` get watch items. The README of the team names this endpoint as the test of the two audiences. ### Each record carries the time of the sighting The `timestamp` of each sighting, sign and change is the time of its video window. Recall returns this time as `occurredAt`. The Assistant gets the records newest first. In its answer, it gives the time since the last sighting of the object. Recall leaves out records with a time after `queryTimestamp`. By default, `queryTimestamp` is the time of the call. A window of video can have a time a little after the clock of the server. For this reason, three recalls of MOM send their own `queryTimestamp`. It is the later of two times: the current time, or the newest time in the journal file of the app. Then the newest window is always in the result. ### A newer sighting replaces the old location Each sighting of an object has the label `{space}:{object}`. The team reports that a newer sighting replaces the old location of the object. In a recall, the old record then has the field `artifact.supersededAt`. The app uses this field in three places: 1. The Comparator leaves the old records out of the last state, so it compares the new window with the current location only. 2. The Assistant gets the old records with the tag `[superseded]` in its prompt. 3. The answer shows frames from current records only. ### MOM deletes the records that are no longer true MOM deletes a record with `DELETE /api/v1/data-points/{id}` in two cases: - If you ask about an object again, the Assistant writes a new watch item for it. Then it deletes the older watch items of that object. So each object has only one current watch item. - If you correct MOM, for example "Remember, we are out of soy milk", the Assistant marks the wrong records. The app stops using them at once, and it writes your correction to memory. Then it deletes the wrong records on past.dev in the background. ### The Assistant waits 2 seconds at most for a recall Each record goes first to the journal, a file of the app on Modal, and then to past.dev. When you ask a question, the Assistant starts the recall and waits 2 seconds at most. If the recall takes longer or fails, the answer uses the journal copy. The app also adds journal records that the recall did not return yet, with the tag `pending`. The team gives the reason in its README. A new record needs some time before recall can return it, and the journal covers that time. ## What memory changed The team did not compare MOM with and without memory. In the speaker notes of the pitch, a baseline for retrieval is on the list of open work. In the technical notes, the team writes that real-world accuracy has no benchmark beyond the live tests. The numbers below come from the video stream of the glasses of Jaspal. The pitch states that this video was filmed with consent. ### The Verifier rejected or held 508 of 647 change claims When you turn your head, the keys go out of the view, and they can look "missing". For this reason, the Comparator does not write a change to memory. It only proposes the change to the Verifier. The Verifier checks each proposed change blind: the model does not see the claim. It asks two questions. Is the object in the earlier frames? Is it in the current frames? Then the Verifier applies the rule for the type of change. For example, a "missing" change needs three things. The object was in the earlier frames, and it is not in the current frames. Its old spot is also visible in the current frames. A "moved" change needs the object in both sets of frames, at a different location. A change goes to past.dev only if it passes its rule with enough confidence. Some "missing" claims are held until they repeat in a second window. So not all of the 508 claims were false. - 647: change claims checked - 508: rejected or held, 79% - 139: changes written to past.dev From the pitch deck of the team. The data is the video stream of the glasses of Jaspal. ### Four identities wrote 1,328 dated records The pitch deck reports 1,328 dated records in past.dev from 4 agent identities. A dated record has the time of its video window. The README reports that each answer uses 20 or more past.dev records. The stage page shows this number under each answer, for example "past.dev, 20 memories". ### The team lists its limits - The data comes from the home of Jaspal and the event hall. The team did not run its scripted benchmark on footage from the glasses. - The 397B model takes about 25 to 55 seconds to process each window. If an object was in view only in the last 30 seconds, it can be absent from memory. - MOM can report an object at the edge of the view as missing. The detection of changes also varies between runs. ## Try MOM 1. Watch the [demo video](https://www.youtube.com/watch?v=5mYZRG-iHpY) (1 min 55 s). 2. Look at the chip under each answer. It gives the number of records from past.dev in the prompt of the answer. 3. Read the [documentation about access control](/docs/memory-api/access-control). It shows how to give each agent an identity, and each group of readers a fixed audience. To give agents one dated memory, start with the [quickstart of the Memory API](/docs/memory-api/quickstart). Thank you to Jaspal for building with past.dev at the hackathon.