PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

PINT. A Unified Model with Shared Predictive Intent for Robot Manipulation. By Xinyu Guo and four others, from the Institute of Automation of the Chinese Academy of Sciences, the University of Chinese Academy of Sciences, and three other institutions. Published on the twelfth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Generalist robot manipulation can draw on two kinds of pretrained model. Vision-language models, or VLMs, bring semantic reasoning. Video generation models, or VGMs, bring world modeling. Existing unified models combine them by fusing the two directly through shared attention at multiple layers. The authors argue that this kind of fusion can make it harder to draw on what each model is specially good at, because the two are blended rather than allowed to keep their own strengths.

PINT is a unified model that coordinates three things, semantic reasoning, world modeling and action generation, through one shared representation called Predictive Intent. It works at the level of chunks, and it keeps expert-specific pathways intact. Rather than mixing the experts layer by layer, the experts meet through this single shared intent.

To form that representation, PINT inserts learnable Intent Tokens into the VLM sequence. The resulting Predictive Intent conditions both the VGM and an Action Expert, or AE. From the VGM's side, the only thing replaced is its prompt conditioning. The VGM is kept at inference, and the AE reads its representations through one-way attention.

PINT does not prescribe what Predictive Intent should encode. It learns it jointly from the native prediction objectives of all three parts: the VLM, the VGM and the AE, each of which offers a complementary view of the same upcoming stretch of task execution. Training runs in two stages. Stage 1 freezes the VGM, so that predicting future video shapes Predictive Intent with task-relevant future dynamics. Stage 2 fixes the learned Intent and adapts the VGM and the AE.

Without embodied pretraining, PINT reports strong in-distribution performance and a substantial improvement out of distribution. On RoboTwin two point zero Hard, it reaches fifteen point five percent, against one point eight percent for Fast-WAM and three point two percent for StarVLA.

On LIBERO-Plus, it reaches seventy-nine point five percent. Fast-WAM sits at fifty-one point five percent there, and StarVLA at seventy-five percent.

On five real-world bimanual tasks, PINT reaches sixty-eight percent success in distribution and twenty-four percent out of distribution. Pretrained Motus scores thirty-five percent in distribution and nine percent out of it. Pretrained pi zero point five scores sixty-seven percent in distribution and thirty-five percent out of it.

The design choice is to make the experts share a goal rather than a set of layers. Each keeps its own pathway, and what passes between them is a compact statement of what is about to happen, learned from all three of their objectives at once. The simulated out-of-distribution gains are where the authors place their strongest evidence, and on the real robot the comparison with pretrained models runs in both directions, with PINT ahead of pi zero point five in distribution and behind it out of distribution.