APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants

APM-Bench. Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants. By Jianguo Huang and eleven others, from Shanghai Jiao Tong University, the Eastern Institute of Technology, Ningbo, and five other institutions. Published on the twenty-ninth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

To serve as real-world personal assistants, streaming video models need persistent memory: a way to retain past experiences for later use. Existing streaming benchmarks and methods often focus on individual continuous videos, or on short clips. Real interactions are often intermittent. Memory has to persist across the interruptions, and that is the case existing evaluations overlook. A personal assistant does not watch one unbroken video. It sees a morning, then nothing, then an afternoon, then the next day, and the question a user asks today may depend on something seen last week.

APM-Bench reformulates real-world streaming interaction as multi-session life trajectories. Models use persistent memory to answer questions about past sessions, and to provide proactive responses, all while keeping the interaction real-time. That places several demands on memory. It must be storable. It must retain information selectively. It must be injected at the right time. And it must remain efficient. Because finite storage may leave required evidence unavailable, assistants should also recognize when evidence is missing. An honest admission that something was not kept is better than a confident answer built on nothing. Each of those demands becomes something the benchmark can test.

The benchmark contains five hundred and forty-nine sessions, one hundred and four trajectories, and two thousand seven hundred and nineteen candidates, spanning both objective and open-ended questions. Each session is a video with fine-grained annotations, and sessions within a trajectory revolve around related activities. That relatedness is what makes memory necessary. A question in a later session can reach back to something that happened in an earlier one.

The evaluation is systematic. General video models are tested under different memory protocols. A range of specialized streaming memory systems are tested too. And models are checked for whether they acknowledge insufficient evidence, rather than answering anyway.

The evaluation reveals a clear trade-off among utility, latency and storage. Existing methods still struggle to achieve three things at once: reliable long-term recall, low overhead, and effective proactive assistance across sessions. Utility, latency and storage pull in different directions, and among the methods evaluated, none yet satisfies all three together. For an assistant that must answer in real time, from a store that cannot grow forever, that is the central difficulty.

Memory that survives an interruption is a different problem from memory within one long video, and APM-Bench is built around that difference. The authors offer it as a comprehensive testbed for developing and comparing persistent memory systems under realistic streaming conditions, in the hope that future work will weigh utility, latency and storage together toward more practical assistants.