Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation

Vision-Language-Delta-Action. Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation. By Lingsen Zhang and seven others, from the Harbin Institute of Technology, Shenzhen, Baidu, and one other institution. Published on the nineteenth of October. Its code repository is worked out: about one thousand five hundred source files, with a readme and a licence.

The starting point is a reframing. Robotic manipulation is not just turning observations into actions. It is making a state change, specified by a task, actually happen in the physical world.

Seen that way, both current families of method carry a cost. Vision-language-action methods, or VLA, learn a direct mapping from observation to action, and that imposes a heavy learning burden because of the distance between understanding a scene semantically and executing an action. World-action models, or WAMs, improve control by predicting the future, but they model complete future states, and so they spend capacity on static appearance and on changes that have nothing to do with the action.

Vision-Language-Delta-Action, or VLDA, sits between the two. It replaces full future prediction with compact modelling of the transitions that matter for acting. At its core is a compact variable called Delta, which captures latent changes in the state of the world. Delta is the interface: vision-language understanding feeds into it, and action generation draws from it.

Delta is learned through one unified objective with two parts. The Local Transition Filter, or LT-Filter, captures task-relevant changes centred on the manipulation, while suppressing the bias toward reconstructing static content. The Global Transition Anchor, or GT-Anchor, adds a coarse regularisation at the level of the future, so that the overall direction of the state change is preserved. One part keeps Delta focused, the other keeps it pointed the right way.

A third component, Delta-Driven Action Refinement, or DDAR, grounds Delta in the action manifold and injects it as a structured seed that guides action denoising.

VLDA is implemented within LoongForge, an open-source training framework for large-scale foundation and embodied models, whose modular and distributed infrastructure supports a unified two-stage optimisation.

In experiments, VLDA improves long-horizon consistency, robustness, and generalisation. The authors read these results as validating an actionable Delta as an alternative both to predicting actions directly and to modelling the full future state.

Two familiar options frame this work. Predict the action straight from what you see, and the model has a long way to reason. Predict the whole future, and it spends effort on what does not move. VLDA proposes a third path: predict only the change the task asks for, and act from that.