World Observer: Joint Actor-Observer Generation for Persistent World Modeling

World Observer. Joint Actor-Observer Generation for Persistent World Modeling. By Hyunwook Choi and six others, from KAIST AI. Published on the first of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Video world models simulate how an environment evolves in response to an agent's actions, but they are actor-centric. They see what the actor sees. Once an object leaves that view, the model has no direct evidence of how it is changing, and when the object comes back into frame its state and dynamics are often lost. Think of a car that drives out of frame: it ought to come back further along, not frozen where it was last seen or changed into something else. The question the authors pose is how a world model could keep watching the parts of the world the actor is not looking at.

World Observer answers by separating observing from acting. It jointly generates two kinds of view. A perspective actor covers the agent-centric view, as before. Alongside it, one or more panoramic observers watch selected regions of the world. An object that walks out of the actor's view keeps evolving visibly inside an observer, so when it re-enters, its updated state is already there to be shown. The actor no longer has to carry the whole world on its own view. Its job is to act, and the job of keeping the rest of the world going is handed to the observers.

The actor and the observers are grounded by warping from a shared panoramic source, which gives them explicit geometric correspondence with each other. For the moment of re-entry, an Observer Sink holds high-resolution perspective references, used to restore fine appearance detail. Because observers are not tied to the actor, they can be placed freely across a scene, multiplied across locations for broader coverage, and driven by control signals that steer how the unseen parts of the world evolve. That last point turns the observers from passive witnesses into a second channel of control, over what happens where the actor is not looking.

Evaluating something that happens off screen required new tools, so the authors introduce world-space metrics and a benchmark spanning real and synthetic scenes. On it, World Observer substantially improves out-of-view dynamics, while staying competitive in visual fidelity, camera control and adherence to 3D structure. In other words, watching more of the world does not come at the cost of how the actor's own view looks or how well it follows the camera.

Persistence, on this account, is not something to be remembered after the fact. It comes from never letting the unseen parts of the world go unrendered. An observer keeps generating what the actor cannot see, so there is something consistent waiting when the actor turns back.