Vision-Language-Action or World-Action? Just Action Models

Vision-Language-Action or World-Action? Just Action Models. By Yonghao Yu and three others, from the University of Tokyo and the National Institute of Informatics. Published on the twenty-seventh of October.

Two communities in robot learning have been arguing about where a control policy should get its pretrained knowledge. The vision-language-action, or VLA, camp inherits it from vision-language models. The world-action model, or WAM, camp inherits it from visual generators. The authors point out that this debate quietly bundles together three choices that can be separated. The first is how the backbone is initialized. The second is what objective is applied to future observations. The third is whether the machinery needed for that future loss is kept around at inference time. Because all three move at once when you swap a VLA for a WAM, any gap between the two cannot simply be pinned on one of them. The comparison, as usually run, is confounded.

Pull the three choices apart, and the two families turn out to share a common framework. The authors call it the Just Action Model, or JAM. It is a pretrained visual backbone paired with an action expert, trained with two losses: an action loss, and what they call a foresight loss. The backbone can come from either camp. What makes it a single framework is that the future supervision no longer depends on generating images.

The foresight loss works on the features of the current observation. Those features are trained to predict frozen encodings of the future that was actually recorded, through a prediction head that is thrown away once training ends. So no visual generation happens at any point, neither during training nor during inference, and nothing extra is carried into deployment. Since the target is a frozen encoding rather than a picture, the backbone can be chosen freely. The study proceeds in two steps. First it compares objectives inside fixed policy architectures, holding everything else still. Then it takes the foresight recipe and evaluates it across different backbone families.

The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and on LIBERO-Plus, and reach similar or slightly higher success rates on RoboTwin. Against FastWAM specifically, success on LIBERO rises from ninety-seven point zero percent to ninety-eight point two percent.

On LIBERO-Plus it climbs from fifty-five point three percent to sixty-four point five percent, the largest of the three gains. On RoboTwin it moves from ninety-one point two percent to ninety-one point four percent. Cost moves as well. With the foresight targets cached, JAM cuts the estimated online training compute per sample per update by seventy-three percent relative to FastWAM.

Two conclusions come out of this. Future supervision improves control without any visual generation and without additional machinery at inference. And strong foresight performance is reachable from either starting point, a generator-initialized backbone or a vision-language one. Read that way, the question in the title dissolves. The choice that mattered was never which family a backbone came from, but whether the policy was taught to anticipate what happens next.