PAIR: Bridging Perception and Action in Vision-Language-Action Models

PAIR. Bridging Perception and Action in Vision-Language-Action Models. By Kaixi Feng and two others, from the University of Maryland, College Park. Published on the sixth of October. No code repository is linked to it.

A vision-language-action model, or VLA, takes visual observations and a language instruction and maps them to continuous robot actions. Somewhere in that mapping there has to be a transition. Representations that describe the scene and the instruction must become representations that support generating an action. In many continuous-action VLAs that transition is left implicit. The only thing supervising it, mostly, is the final action-prediction loss at the very end of the model.

PAIR makes the transition explicit. It is a framework that learns a shared perception-action representation, a middle space that sits between what the model sees and what it does. The aim is a set of tokens that keep the task information from perception while already carrying the structure of expert actions, so that action generation starts from something action-shaped.

Training adds two components. A Masked Action Autoencoder encodes expert action chunks into horizon-aligned Action Latent Tokens. Separately, a Bridge Module extracts task-relevant features from the current visual-language representations. PAIR aligns those features with the Action Latent Tokens, and the result is a set of Bridge Tokens that preserve task information and capture the structure of expert actions.

The Bridge Tokens are then projected into the action-token space and injected into the initial Action Tokens. That gives the Action Expert an action-ready starting point to refine, instead of a blank one. At inference the autoencoder is removed entirely. The Bridge Tokens are generated only from the current observation and instruction, since expert actions are no longer available.

Experiments cover LIBERO, LIBERO-Plus and CALVIN ABC-D, with gains for both models evaluated, OpenVLA-OFT and VLA-Adapter. On LIBERO-Plus, PAIR raises the success rate of VLA-Adapter from fifty-nine point one percent to sixty-four point two percent. On CALVIN, it lifts the average completed sequence length of VLA-Adapter from four point four two to four point five three.

On the robot itself, across seven real-world tasks, PAIR raises the success rate of OpenVLA-OFT from fifty-one point four percent to sixty-five percent. Representation analyses add a further observation. Bridge Tokens retain task information while making continuous-action information accessible before the Action Expert has refined anything.

The argument is about where supervision lives. When the only signal is the loss on the final action, the handoff from perception to action is left to form however it forms. PAIR gives that handoff its own target, shaped by expert actions during training and produced from observation alone at inference. The authors read their results as support for a shared intermediate representation as a useful interface between perception and action in continuous-action VLAs.