EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks
Lizhou Liang, Xinyu Zhong, Miao Pan, Xiaohe Zhou, Xuanyu Liu, Qinfeng Li, Peng Li, Jintao Chen and 2 others
Zhejiang University · Central South University · Institute of Software, Chinese Academy of Sciences
Published September 23, 2026 · on the feed September 29, 2026 · ▲ 39 · arXiv 2609.28236
The abstract
Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter change. Yet current agents struggle to maintain such memory reliably. Our analysis traces this limitation to four key deficiencies: weak fine-grained visual memory, unreliable dynamic world-state tracking, failing to record world state revealed by interaction outcomes, and limited generalization from prior experience. However, existing benchmarks do not directly assess these memory capabilities during long-horizon embodied interaction. To address this gap, we introduce EmbodiedMemory-Bench (EMem-Bench), comprising 2,554 interactive episodes across four task families. EMem-Bench requires agents to build and update memory from interaction history, then use it to complete a later task by acting in the environment. We further present Embodied-Memorizer (EMem), an external memory system that organizes embodied experience into spatial, event, and scene memories. We also train EMem-8B, an 8B policy that manages and uses these memories. We evaluate a diverse range of open-source and proprietary MLLMs and representative multimodal memory systems. Results show that current models remain weak and uneven across the four challenges. Under matched backbones, EMem achieves the best overall performance among the evaluated memory systems and improves both open-source and proprietary models, while EMem-8B further improves over its backbone. Project page: https://zju-omniai.github.io/EmbodiedMemoryBench/
The Brief
2:33A short podcast summary of the paper. Written and voiced by AI.
Transcript
EmbodiedMemory-Bench. Benchmarking Embodied Memory for Long-Horizon Embodied Tasks. By Lizhou Liang and nine others, from Zhejiang University, Central South University, and one other institution. Published on the twenty-third of September. No code repository is linked to it.
An agent working in the physical world over a long stretch has to keep track of its surroundings. It observes, it acts, things change, and it has to retain and keep updating what it knows through all of that. Current agents struggle to do this reliably. The authors' analysis traces the weakness to four specific deficiencies. Fine-grained visual memory is weak. Tracking a world state that changes over time is unreliable. Agents fail to record the parts of the world state that are revealed only by the outcomes of their own actions. And they generalize poorly from prior experience. Existing benchmarks, the paper says, do not directly test any of these memory capabilities during long-horizon embodied interaction.
EmbodiedMemory-Bench, shortened to EMem-Bench, is built to test exactly those abilities. An agent must build and update a memory from its interaction history, and then use that memory to complete a later task by acting in the environment. Remembering is not scored by a quiz. It is scored by whether the agent can do something that depends on what it remembered.
The benchmark holds two thousand five hundred and fifty-four interactive episodes, spread across four task families. Alongside it, the authors present a memory system of their own, Embodied-Memorizer, or EMem. It is an external memory that organizes embodied experience into three kinds of store: spatial memories, event memories and scene memories. They also train EMem-8B, an eight billion parameter policy that manages and uses these memories.
The evaluation spans a diverse range of open-source and proprietary multimodal large language models, along with representative multimodal memory systems. The headline is that current models remain weak, and unevenly so, across the four challenges: strength on one does not carry to the others. Under matched backbones, EMem achieves the best overall performance among the memory systems evaluated, and it improves both open-source and proprietary models when attached to them. The trained EMem-8B improves further over its own backbone.
The four deficiencies give the benchmark its structure, and they also give it a use beyond a single score. A model that fails here fails in a particular way, whether that is losing visual detail, losing track of a change, or missing what its own actions revealed, and the uneven results suggest those failures are distinct problems rather than one.
Paper Quest · 7 chapters · about 36 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A robot that forgets is stuck reacting to whatever is in front of it. Start with that.
You are building a home robot that works through long chores. It sees a lemon on the counter, someone moves a knife, a drawer it tries turns out to be locked.
The road ahead
- 1 · The problem ~7 min Robots on long chores keep acting as if they forgot what they saw and learned, and no current test directly checks that remembering while they act.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min A benchmark scores memory by whether the agent can act on it, across four kinds of forgetting, plus a memory system with three stores.
- 4 · Choose your own research ~3 min
- 5 · How it works ~3 min
- 6 · The experiment ~6 min You’ll guess: Tested on long-horizon embodied memory across four kinds of challenge, how did current open-source and proprietary multimodal models perform?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.