Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence

Magic-W0. A Structured World-Action Foundation Model for Physical Intelligence. By Xuhua Chen and seventeen others, from Magiclab Robotics. Published on the third of October. Its code repository is a placeholder: a readme, and nothing to run yet.

World-action models, or WAMs, give a robot policy a model of how its environment changes in response to actions. Existing approaches mostly get there by reconstructing future observations or by generic latent prediction. What they lack is a world representation that is structured, oriented toward control, and tightly coupled with the generation of actions. Predicting what the future looks like is not the same as representing what an action does.

Magic-W0 is a world-action foundation model that jointly models structured physical state evolution and continuous actions. It represents each interaction as a Structured World Transition with three parts. Current State combines vision-language context with Current 3D Geometry. Transition is represented by 3D Motion, which captures the three-dimensional changes an action induces. And Future State is represented by Future Semantics, describing the outcomes that matter for the task.

To couple prediction with control, the authors propose a layer-aligned architecture for world-action interaction. It runs in both directions. Evolving action hypotheses condition the prediction of the world transition, while the predicted world representations continuously inform how actions are generated. Magic-W0 is pre-trained on a broad mix: large-scale egocentric human manipulation, UMI data, real-robot data, and simulation. Latent supervision for geometry, 3D motion, and future semantics comes from pre-trained visual models.

Inference-time interventions probe whether the structure is doing real work. They show that the structured world representations respond systematically when candidate actions change, and that information related to the action propagates through the shared 3D representations into the future semantic predictions. On RoboDojo-Sim, Magic-W0 reaches an average Score of twenty-seven point one zero, the highest among the compared WAMs. Across multiple real-robot tasks it also performs strongly after fine-tuning with limited downstream data, which the authors read as support for generalisation and rapid adaptation.

The contribution is a choice about what a world model should predict. Instead of whole future frames, Magic-W0 predicts geometry, motion and task-relevant outcomes, and lets those predictions and the actions shape each other layer by layer. The interventions are what show the structure is used, not just present.

It helps to follow one action through the model. A candidate action enters as a hypothesis. It conditions the predicted 3D Motion, the change the action would cause. That motion, carried through the shared 3D representations, shapes the Future Semantics, the outcome that matters for the task. And the predicted world flows back to inform the action. The interventions show this path working as described, and the benchmark and real-robot results show it paying off downstream.