Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation
Weijie Wang, Haomin Zhang, Ye Li, Akide Liu, Zeyu Zhang, Haoyu Zhao, Yefei He, Donny Y. Chen and 2 others
Zhejiang University · Monash University · UC Berkeley · Adelaide University
Published October 14, 2026 · on the feed October 15, 2026 · ▲ 344 · arXiv 2610.20571
The abstract
Long-form agentic video generation must turn a screenplay into shots that remain spatially and temporally consistent across nonconsecutive requests. Passing prompts or recent frames between shots does not provide a shared spatial frame for revisiting a scene or a time-indexed record for replaying an earlier event. We present Any Time, Any Space (ATAS), which compiles a screenplay into shot requests with explicit location, camera, and event-time references. Before generation, it retrieves relevant evidence from a scene memory of registered static geometry and an event memory of dynamic observations indexed by scene, event, and time. After generation, ATAS reconstructs and registers the clip. It updates memory only when review and state checks accept the observation, while film delivery remains independent of memory admission. Across four video generators, ATAS improves all reported cross-shot consistency measures and achieves the highest aggregate benchmark score for each backbone. It also outperforms the compared agents and 3D-guided methods on target-image spatial revisit, leads three event-recapture criteria, and receives the highest human ratings on all six criteria. The video results and code are available on our project page: https://lhmd.top/atas/.
The Brief
2:14A short podcast summary of the paper. Written and voiced by AI.
Transcript
Any Time, Any Space. A World-State-Centric Framework for Agentic Video Generation. By Weijie Wang and nine others, from Zhejiang University, Monash University, and two other institutions. Published on the fourteenth of October. Its code repository is worked out: about three hundred and fifty source files, with a readme and a licence.
Long-form agentic video generation has to turn a screenplay into shots, and those shots must stay consistent in space and in time even when the requests for them are not consecutive. A story may revisit a scene it left earlier, or replay an event that already happened. Passing prompts or recent frames from one shot to the next does not support either. It gives no shared spatial frame for revisiting a scene, and no record indexed by time for replaying an earlier event.
Any Time, Any Space, or ATAS, puts an explicit world state at the centre. It compiles the screenplay into shot requests, and each request carries explicit references to a location, a camera, and an event time. Those references point into two memories. A scene memory holds registered static geometry. An event memory holds dynamic observations, indexed by scene, by event, and by time.
Each shot passes through memory on both sides of generation. Before it, ATAS retrieves relevant evidence from the two memories. After it, ATAS reconstructs the new clip and registers it. Memory is updated only when review and state checks accept the observation. Film delivery, though, does not wait on that decision: whether a clip enters memory and whether it reaches the film are kept independent.
The framework was run across four video generators. With each one, ATAS improves every reported measure of cross-shot consistency, and earns the highest aggregate benchmark score for that backbone. On revisiting a target image's space, it outperforms the compared agents and the 3D-guided methods. It leads on three criteria for recapturing events. And in human ratings, it scores highest on all six criteria. Video results and code are on the project page.
The design treats a screenplay less like a sequence of prompts and more like queries against a world that persists between shots. Space is held as geometry, time is held as an indexed record of events, and a gate on what gets written keeps that world from absorbing clips the checks reject.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A film is many shots, made one at a time. The hard part is remembering what came before.
You are turning a screenplay into a film, one short generated shot at a time. The story shows an event in a room, cuts away to another place, and later returns to that room.
The road ahead
- 1 · The problem ~7 min When a generated film returns to a room or replays an event many shots later, passing along recent frames or prompts cannot bring the old room back.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Keep a lasting world record in two memories, look it up before each shot, and admit new clips only after they pass checks.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Each shot runs compile, retrieve, generate, reconstruct and register, then a checked memory update that later shots read.
- 6 · The experiment ~6 min You’ll guess: Run with four different video generators, did ATAS improve every reported cross-shot consistency measure?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.