Native Action-Prior Learning from Videos for World Action Models

Native Action-Prior Learning from Videos for World Action Models. By Zhaochong An and twelve others, from Meta AI, the University of Copenhagen, and two other institutions. Published on the second of October. No code repository is linked to it.

World action models put two things together: predicting how the visual scene will unfold, and predicting what the robot should do. Their scalability is held back by data. They need robot trajectories annotated with actions, and those are scarce. Videos with no action labels are plentiful and carry rich evidence about how interactions unfold, but existing approaches use them only indirectly. Either they pretrain visual representations that must later be adapted for control, or they infer latent actions that then have to be grounded to real robot commands. In both cases something stands between the video and the policy.

NAVA-WAM removes that intermediate step. Its idea, native action-prior learning, is to pretrain the action policy itself directly from observation-only videos. There is no representation to transfer into control afterwards, and no separate latent-action model to train and ground.

Training runs in two stages. In the first, the model sees only videos. It is supervised by flow matching on future video, over the visual transitions, and that supervision is propagated through a joint attention structured around transitions, so that it reaches and optimises the Action-DiT. That is where the action-relevant priors are learned, even though no action label has been seen.

In the second stage, demonstrations that do carry action labels are used to post-train the same Action-DiT for robot control, now with joint flow matching over video and action together. Here the attention is asymmetric. It decouples the visual branch from the iterative denoising of actions, and that decoupling is what allows inference to run on actions alone, which makes it efficient.

Extensive experiments show NAVA-WAM consistently outperforming prior approaches, both in distribution and out of distribution. The authors also report strong efficiency with respect to action labels, meaning good results from fewer of them, and effective generalisation on a real robot. The abstract gives these as qualitative statements rather than specific figures.

The claim the authors draw is that action policies can be pretrained directly from videos that never recorded an action, and that this native route is effective. If it holds, the bottleneck shifts from collecting labelled robot trajectories to collecting ordinary video, which is the scalable path beyond action-labelled data that the paper is arguing for.