WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
Haiyu Zhang, Wenqiang Sun, Tengfei Wang, Junta Wu, Jun Zhang, Yunhong Wang, Yu Qiao, Chunchao Guo
Published September 28, 2026 · on the feed September 29, 2026 · ▲ 31 · arXiv 2609.35560
The abstract
Interactive world models require responding in real time to versatile controls and maintaining long-horizon consistency. However, modeling heterogeneous controls remains difficult, while explosive contexts and unstable distillation impede achieving both long-horizon consistency and real-time responsiveness. In this paper, we present WorldPlay2, an interactive world model that couples a factorized hybrid control interface with a co-design of compressed memory and stable distillation. 1) Our factorized hybrid control interface integrates frame-aligned action control with structured semantic control that explicitly disentangles scene appearance, character identity, and dynamic semantic events, thereby facilitating effective control learning. 2) To achieve efficient long-horizon modeling, we compress historical contexts into compact memory tokens shared by the autoregressive student and the bidirectional teacher. This design enables clip-wise, memory-conditioned score evaluation instead of jointly processing an entire long rollout, substantially reducing distillation overhead. 3) We further propose Stable Forcing, which initializes the autoregressive student via a few-step strategy and leverages full-rollout replay to preserve the quality of long-horizon rollouts, ensuring robust and stable distillation. Extensive experiments demonstrate the strong generalizability of our model and its superior performance compared to existing methods.
The Brief
2:18A short podcast summary of the paper. Written and voiced by AI.
Transcript
WorldPlay2. Extending Real-Time Interactive World Models in Control and Horizon. By Haiyu Zhang and seven others. Published on the twenty-eighth of September. Its code repository is a placeholder: a readme, and nothing to run yet.
An interactive world model has to answer versatile controls in real time and stay consistent over a long horizon. Both are hard. Controls come in heterogeneous forms that are difficult to model together. And on the horizon side, contexts grow explosively and distillation is unstable, which together stand in the way of getting long-horizon consistency and real-time responsiveness at once.
WorldPlay2 answers with two pieces. The first is a factorized hybrid control interface. The second is a co-design of compressed memory and stable distillation, built so that the memory and the distillation work in concert.
The control interface combines frame-aligned action control with structured semantic control. The semantic part explicitly separates scene appearance, character identity and dynamic semantic events, which is meant to make control easier to learn. Frame-aligned actions cover the moment-to-moment steering. The semantic channel covers what the world contains and what happens in it, with each of those three factors given its own handle rather than being tangled into one signal.
For the long horizon, historical context is compressed into compact memory tokens, shared by the autoregressive student and the bidirectional teacher. That sharing lets scores be evaluated clip by clip, conditioned on memory, instead of processing an entire long rollout at once, and it substantially cuts the overhead of distillation. On top of this sits Stable Forcing. It initializes the student through a few-step strategy and uses full-rollout replay to keep long rollouts at high quality, so that distillation stays robust.
Extensive experiments are reported to show the model's strong generalizability, and performance superior to existing methods. The abstract states these as conclusions without attaching figures, so the size of the margin, and which methods it is measured against, are left to the paper itself.
Control and horizon are treated here as separate problems with separate fixes. The controls get factored. The past gets compressed into something student and teacher can both read, and the student is started and replayed in a way that keeps long rollouts from degrading. Real-time response and long-horizon consistency are meant to come from that combination.
Paper Quest · 7 chapters · about 37 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A world you can steer in real time has to remember where you've been. Memory is expensive.
You are building a generated world a user can steer in real time: walk around, turn the camera, make things happen. Some controls need frame-by-frame precision, others describe events over longer stretches.
The road ahead
- 1 · The problem ~7 min A steerable generated world must answer mixed controls instantly and stay consistent for a long session, but long videos make teaching a fast model very costly and unstable.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Squeeze history into a few memory tokens that student and teacher share, score short clips against that memory, and keep the student stable with a warm start and full rollouts.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min First warm-start the step-by-step student, then compress its rollout history into memory tokens, then let the teacher score each clip against that memory.
- 6 · Field test ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.