WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

WorldMemBench. A Comprehensive Benchmark for Memory of Video World Models. By Nan Zhou and twelve others, from Zhejiang University, Fudan University, and two other institutions. Published on the thirteenth of October. Its code repository is worked out: about seventy-five source files, with a readme and a licence.

Video models now support longer and more realistic interactive experiences, and as they do, a new requirement comes forward: the world has to stay consistent with its own history. That means two things. What was already seen must be preserved, and the world must evolve coherently over time. The authors argue that this world memory is a critical capability, one to be evaluated alongside the familiar measures of visual fidelity and video duration. A video can look excellent frame by frame and run for a long time, and still forget what was in the room a moment ago, or let a story stall where it should have moved on. Fidelity and duration say nothing about either failure, which is why memory needs a test of its own.

WorldMemBench splits memory in two. Static memory is about retention: of a person's identity and appearance, of an object's appearance and geometry, and of environmental content and geometry. Dynamic memory is about change: whether events progress as they should, and whether states persist once reached. The first kind asks whether the world stayed put where it should have. The second asks whether it moved on where it should have, and then held its new state.

There are two hundred and sixty test cases. The camera is deliberately sent away from a scene in a controlled way and brought back, and some trajectories revisit the same place several times. Static content is then compared against fixed initial references, while dynamic events are compared against the state transitions that ought to have occurred. Scoring combines appearance and geometry metrics with assessments specific to each event. Video quality and how accurately the model follows the camera are reported too, as complementary diagnostics rather than the main measure.

Nine representative models were evaluated. Two findings stand out. Strong static retention does not imply reliable dynamic state evolution: a model can remember how things looked and still get wrong how they should have changed. And repeated revisits reveal memory degrading beyond the first return, a failure a single return would not catch. Coming back once and finding things intact is no guarantee of finding them intact the second or third time. Human studies support the alignment between the automatic evaluations and human judgments.

WorldMemBench is offered as a systematic testbed for diagnosing where memory fails and for guiding work towards video worlds that are persistent and temporally coherent. Its sharpest lesson lies in the separation it draws: remembering appearances and tracking how a world moves on are different abilities, and they need to be measured apart.