Precise Editing and Flexible Referencing for Interactable Worlds
Xinyao Liao, Xianfang Zeng, Zhu Liang, Zhoujie Fu, Qianxun Xu, Jiachi Liu, Gang Yu, Guosheng Lin
Nanyang Technological University · StepFun
Published September 28, 2026 · on the feed September 29, 2026 · ▲ 19 · arXiv 2609.34470
The abstract
We present EditWorld, a video world model for precise editing and flexible referencing in interactable worlds. Existing video world models primarily focus on navigation, letting users explore generated worlds but offering limited control over how existing world content is modified. EditWorld extends world modeling from exploration to precise modification by streaming editing instructions and reference images during autoregressive generation. To support these capabilities, EditWorld introduces Gated Causal Attention for temporally varying editing conditions and reference images, together with a Sparse Context mechanism that maintains a bounded historical context for long-horizon inference. We further adopt joint autoregressive and bidirectional training with annealed self-resampling, and construct a dedicated data synthesis and annotation pipeline that provides supervision for world editing. We also present WBench-Editing to systematically evaluate streaming world editing capabilities. EditWorld achieves the best overall performance on WBench-Editing with an overall score of 73.8 and an editing score of 80.0, substantially outperforming existing methods on editing-related metrics. https://github.com/leoisufa/EditWorld
The Brief
2:05A short podcast summary of the paper. Written and voiced by AI.
Transcript
Precise Editing and Flexible Referencing for Interactable Worlds. By Xinyao Liao and seven others, from Nanyang Technological University and StepFun. Published on the twenty-eighth of September. Its code repository is public and real: about thirty source files.
Existing video world models mostly focus on navigation. They let a user explore a generated world, but they offer limited control over how the content already in that world gets modified. Walking through a place and changing it are different capabilities, and it is the second that has been underserved.
EditWorld is a video world model built for precise editing and flexible referencing in interactable worlds. It extends world modeling from exploration to precise modification by streaming editing instructions and reference images into the model while autoregressive generation is still under way. An edit can arrive mid-stream, and the world carries on from there with the change in place.
Two architectural pieces make this work. Gated Causal Attention handles editing conditions and reference images that vary over time. A Sparse Context mechanism keeps a bounded window of history, so inference can run over long horizons without the context growing without limit. Training is joint, autoregressive and bidirectional, with annealed self-resampling. Supervision for world editing comes from a dedicated pipeline that synthesises and annotates the data. That last piece matters because an editing model needs examples of worlds being changed, not only worlds being explored, and the authors build those examples themselves rather than assuming they exist.
Measuring edits in a live stream calls for its own yardstick. To evaluate streaming world editing systematically, the authors also present WBench-Editing. On it, EditWorld has the best overall performance, with an overall score of seventy-three point eight and an editing score of eighty point zero, substantially outperforming existing methods on the metrics tied to editing.
The move here is from a world you can only look around to a world you can act on while it is still being generated. Instructions and references are fed in as the stream goes, and a purpose-built benchmark checks whether those changes land.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Walking through a generated world is solved, more or less. Changing it is not.
You work on video generators that build a world frame by frame as a user moves through it. Walking around works well; users can steer the camera and act.
The road ahead
- 1 · The problem ~6 min Video generators let you walk through a world, but they give you little control over changing what is already in it.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Stream edit instructions and reference images into the generator mid-video, use a gated attention to apply each at the right time, and keep history bounded.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Build editing training videos, train the model with joint causal and bidirectional objectives plus self-resampling, then stream edits in while generating.
- 6 · The experiment ~6 min You’ll guess: On the new WBench-Editing benchmark for streaming world editing, what editing score did the EditWorld video world model reach?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.