Agentic World Action Models

Agentic World Action Models. By Zhuoyang Liu and seventeen others, from UC Berkeley, NVIDIA, and one other institution. Published on the twenty-first of October. No code repository is linked to it.

Two lines of robotics work each hold half of something. Recent world action models provide visual foresight and future prediction for robot manipulation: they can imagine what will happen next. But they lack explicit mechanisms for reusing skills compositionally. Agentic robotics is the mirror image. It combines code-based planning and reasoning with reusable skills, but it often lacks predictive grounding in physical dynamics. One side can predict but not compose. The other can compose but not predict.

AW-0 is an agentic world action model framework that joins the two halves. It integrates code-based planning with joint prediction of video and action. Code, future video and robot actions are modelled together, as three coupled generative streams inside one unified mixture-of-transformers architecture. Rather than a planner handing instructions to a separate predictor, all three are generated by one model.

Each stream has its own expert. A code expert generates parameterized skill routines and control logic, while video and action experts generate future visual states and low-level motor commands. Code plays two complementary roles. Within the world model, it structures multi-step prediction and control. Beyond it, it represents skills as reusable programs for subsequent agentic execution, so a routine written once can be called again later.

Data comes from an automated curation pipeline. It labels robot and human video trajectories with aligned programs, and consolidates routines that recur across them into reusable skill libraries. That is how the code stream gets something to learn from.

Evaluations on RoboTwin 2.0 and on physical bimanual manipulation tasks show improved task performance over VLA and WAM baselines. Alongside that come gains in language following, in semantic reasoning, and in robustness.

In this design, code is the joint between the parts. It is what the planner writes, what structures the world model's predictions, and what gets saved as a skill for next time. The authors' conclusion is that agentic code connects planning, prediction and execution within world action models, closing the gap between the two lines of work the paper began with.