VLA-Δ: Learning Physical State Transition for Vision-Language-Action Models

VLA-Δ. Learning Physical State Transition for Vision-Language-Action Models. By Qi Lu and seven others, from Zhejiang University and Central South University. Published on the fifteenth of October. Its code repository is worked out: about one hundred and ten source files, with a readme and a licence.

Vision-language-action models carry the semantic knowledge they acquire in vision-language pretraining over into robot control, and they perform strongly across many benchmarks. The weakness the authors point to lies in what these models are asked to predict. Most of them map an observation straight to the demonstrated action, with no explicit supervision of the physical state changes that happen in between. That leaves them prone to overfitting the correspondence between what they see and what was done. Some methods do add an intermediate step, either reasoning or a prediction of the future state, expressed in language, in images, or in a latent space, and use it to guide action generation. None of them explicitly supervises the physical state transition itself.

VLA-Delta is a general framework that makes that transition the target. The physical change between the current state and a future state becomes an explicit learning target, so the model is trained not only on which action to take, but on what that action changes in the world.

Supervising a change requires labels for it. To support the objective, the authors construct a large-scale pretraining dataset and derive physical state-transition annotations for it. Those annotations cover two kinds of information: task-relevant semantics, and visual motion changes. Together they provide more direct supervision for continuous action generation than the demonstrated action does on its own. The transition is a training signal rather than a step at test time. During inference, the model generates actions directly from current observations, with no additional inference overhead.

On the LIBERO and RoboTwin benchmarks, VLA-Delta achieves performance competitive with the state of the art. In real-robot experiments, it outperforms pi zero point five. The code is publicly available.

The argument concerns supervision, not architecture. Earlier approaches guided action with predicted futures in some representation. This one supervises the difference between now and next directly, and then leaves it behind at inference, so whatever the model learned about physical change costs nothing when the robot actually moves.