State World Models: Execute the Rules, Generate the Pixels

State World Models. Execute the Rules, Generate the Pixels. By Chan Yan Ki and seven others, from Tsinghua University and GigaAI. Published on the nineteenth of October. Its code repository is worked out: about two hundred source files, with a readme and a licence.

Existing world models can generate action-conditioned video from a single image, and many of them focus on camera movement, letting a user explore the scene they generate. Navigation alone, though, does not make an action follow persistent rules. The hard case is an action whose outcome depends on state that is invisible in the current frame. Two situations can begin from the same picture and still require different endings, because something the camera cannot see differs between them. A model working only from what is visible has no record of that difference to draw on.

Stateful Harness separates world evolution from video rendering, to support interaction that is stateful and governed by rules. One part maintains an evolving world state under specified rules: that is where the rules are executed. A frozen video model does the rendering, turning each resulting event into a clip. The generated clip is then used to update the world state for the next turn, so the two halves stay in step. The title states the division plainly: execute the rules, generate the pixels.

Each turn runs in a fixed order. The user selects an action, and the harness resolves it against the current state. It then compiles the resulting event into an instruction for the video model. Alongside that instruction, a preview clip from an editable 3D scene supplies spatial and camera guidance. The video model renders, and its clip feeds back into the state. Nothing about the video model itself changes. Everything to do with rules happens in the harness around it.

Evaluating rule compliance needed a benchmark of its own. WorldState-Bench holds fifty counterfactual twin pairs. The two halves of a pair share an initial frame and an action, but differ in hidden state, and therefore in the outcome they require. Success means producing the required ending for both cases in a pair.

Under rule-only prompting, the best of the eleven evaluated video backends manages that in thirty-two percent of pairs. For MiniMax-H3, spelling out the required endings raises the rate from thirty-two to sixty-four percent. The full harness, with the same frozen model, reaches ninety-two percent.

Those three figures read as a sequence. Rules alone leave most pairs wrong. Telling the model the endings doubles the rate. Moving rule execution into a separate world state, and leaving the video model only the job of rendering, takes it to ninety-two percent without touching the model's weights. The authors take this as support for separating rule execution from video rendering whenever world interactions depend on hidden state.