VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models

VLA-Precision. Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models. By Chenyu Su and ten others, from the University of Science and Technology of China, the State Key Laboratory of Precision and Intelligent Chemistry, and three other institutions. Published on the eighteenth of September. Its code repository is worked out: about one hundred and ten source files, with a readme and a licence.

Pretrained vision-language-action models give robots broad manipulation ability, but they remain unreliable on tasks that demand precision and repeatability. One route past demonstrations is real-world online reinforcement learning as a post-training step, letting the robot improve autonomously by trial and error. That route exposes two bottlenecks. Value signals can be unreliable, and unreliable values can induce policy drift. And the overhead of a large model constrains throughput and sample efficiency.

VLA-Precision is a real-world online reinforcement learning framework built to address both. It has two parts: an algorithm called Asymmetric Co-Bootstrapping, or ACoB, aimed at the value problem, and an architecture called ACoB-Stream, aimed at the overhead problem. The asymmetry in the name is about timescales. The policy and the value estimates are bootstrapped together, but different signals lead at different stages of learning.

Early on, intervention-guided behavioural learning takes the lead. It improves policy performance quickly, and in doing so it raises the quality of the online experience being collected. As autonomous experience accumulates, the value side catches up. Global return propagation and local preference ranking progressively calibrate the value estimates, which yields relative action advantages. Those advantages drive policy improvement that is regularized toward a reference, and that regularization is what suppresses drift.

ACoB-Stream is what makes this feasible on large models. It is a closed-loop architecture connecting experience and policy, built on two design principles: invariant-state decoupling and on-demand streaming. It delivers up to ten point nine times improvements in throughput and computational efficiency.

Evaluation covers nine high-precision chemistry tasks, in four categories, on four robot embodiments. VLA-Precision reaches a ninety-eight point three percent mean success rate, in forty-five point eight minutes per task. Episodes last twenty-seven point six seconds. At that pace the system runs at one point two times the speed of vision-language-action baselines, and one point eight times the speed of reinforcement learning baselines.

The framework pairs each bottleneck with its own remedy. Drift is handled on the learning side, by letting guided behaviour lead until the value estimates are calibrated enough to be trusted. Throughput is handled on the systems side, by an architecture whose principles, decoupling and streaming on demand, are what let the algorithm run on a large model at all.