MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories
Guangzhi Xiong, Xinyuan Zhang, Xiao Yang, Hyokun Yun, Kai Zhang, Shiun-Zu Kuo, Hyeonjeong Ha, Xilun Chen and 9 others
Meta Reality Labs · University of Virginia · University of Illinois Urbana-Champaign
Published September 30, 2026 · on the feed October 1, 2026 · ▲ 2 · arXiv 2609.40195
The abstract
Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanning months or years, reprocessing raw clips for every query becomes computationally prohibitive. Memory systems offer a scalable alternative by compacting videos into text representations, but often fail on practical benchmarks: either the memory does not preserve key evidence, or the retriever fails to locate relevant entries due to retrieval competition in growing search spaces. To address these challenges, we introduce MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader. Without training or query-time video access, MemLife improves over the strongest training-free baseline by 4.6--12.0% across four long-horizon benchmarks. To further improve memory quality, we propose MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories. MemOpt consistently improves MemLife by 2.7--5.0% across different video and question distributions, with gains that generalize across writer and reader backbones and memory systems.
The Brief
2:04A short podcast summary of the paper. Written and voiced by AI.
Transcript
MemLife. Curating and Reasoning over Long-Term Egocentric Video Memories. By Guangzhi Xiong and sixteen others, from Meta Reality Labs, the University of Virginia, and one other institution. Published on the thirtieth of September. No code repository is linked to it.
Long-term egocentric video, filmed from the wearer's own point of view, could let a personal AI assistant reason about someone's daily life. The trouble is volume. Once a history runs to hundreds of hours over months or years, reprocessing the raw clips for every question becomes computationally prohibitive. Memory systems are the scalable alternative: compact the video into text, and query that instead. On practical benchmarks they often fail, in one of two ways. Either the memory never kept the key evidence, or the retriever cannot find the right entry, because entries compete with each other in a search space that keeps growing.
MemLife is a multimodal memory system aimed at both failures. It writes memories as first-person text episodes grounded in entities, and it reads them back through an agentic reader that is indexed by time.
On top of that system sits a second contribution, MemOpt. It is a reinforcement learning framework that trains the memory writer itself, with the goal of memories that are faithful, informative, and retrievable. The writer, in other words, is optimised for what the reader will later need.
Even before MemOpt, and without any training or access to the video at query time, MemLife beats the strongest training-free baseline by between four point six and twelve percent across four long-horizon benchmarks. Adding MemOpt improves MemLife consistently, by between two point seven and five percent, across different distributions of video and of questions. Those gains generalize across the backbones used for the writer and the reader, and across memory systems.
The two failures named at the start map onto the two halves of the work. Losing evidence is a writing problem, and MemOpt trains the writer. Losing entries in a growing pile is a reading problem, and the time-indexed reader addresses it. Months of footage become text an assistant can search, rather than video it must watch again.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Months of video, one small question. The wall is remembering the right thing.
You are building an assistant that sees through someone's smart glasses for months. Later they ask: where did I leave my ring last month? How many times did I take my medication this week?
The road ahead
- 1 · The problem ~7 min An assistant watching months of first-person video must answer questions about the past, but rewatching is too slow and text notes can fail.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Write careful time-anchored first-person text episodes, read them with a search agent that also fetches by time, and train only the writer.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Train the writer first, then write grounded first-person episodes, retrieve them with a time-indexed agent, and reason over them in time order.
- 6 · The experiment ~6 min You’ll guess: With no training and no video access at question time, at most how much did MemLife improve over the strongest training-free baseline?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.