ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

ActionPiece. Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models. By Shijie Lian and seven others, from Huazhong University of Science and Technology, Zhongguancun Academy, and six other institutions. Published on the sixteenth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

In autoregressive vision-language-action models, or VLA models, the action tokenizer plays a central role. It decides what targets the policy trains on, and it decides what executable commands are recovered from the tokens the model predicts. Tokenizers are usually judged by pointwise reconstruction metrics, such as mean squared error, MSE. But small individual errors do not tell the whole story. After compression, similar actions can still cluster around one representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. Each action may be reconstructed closely, and yet the differences between actions, which are what make a policy fit its context, can be lost. A small average error, in other words, can hide a policy that has lost the ability to tell one situation's motion from another's.

Two contributions follow. The first is a measure: physical rank consistency, PRC. It asks how well tokenization preserves the ranking of local physical distances after reconstruction: whether actions that were near each other stay nearer than actions that were far apart. Because it is measured on decoded actions, it gives a common reference across token vocabularies and decoder architectures, and it complements pointwise accuracy with a measure of relational fidelity.

The second is ActionPiece, a tokenizer that preserves those physical relationships by jointly supervising representation learning and quantization.

Two objectives do this. Physical rank preservation supervises the ordering of near and far, both in the encoder's feature distances and in the quantized feature distances. Quantization regularization applies the same ordering to the distributions over codeword assignments. Both objectives are added to reconstruction, not substituted for it. The result is a set of discrete action tokens that can be used for standard autoregressive policy learning, and executed through a frozen decoder. Nothing about the downstream policy has to change. The tokens ActionPiece produces are ordinary discrete tokens, and the policy is trained and run on them in the standard way. What changes is what those tokens have been trained to keep.

Under the same policy training setup, built on Qwen3-VL-4B, ActionPiece reaches ninety-four point eight percent on LIBERO, and sixty-eight point eight percent on the unseen LIBERO-Plus. Further evaluations reach seventy-one point nine percent on SimplerEnv, and fifty-one point five percent across levels zero to two of VLA-Arena.

Component ablations show that the two objectives together improve both PRC and policy success. That is the link the paper set out to show: a tokenizer trained to keep physical rankings scores better on the new measure, and the policies built on it succeed more often.

The paper asks a tokenizer to keep relationships, not only values. A reconstruction that is accurate point by point can still flatten the differences between actions that different contexts demand. Measuring how well rankings survive, and training the tokenizer to keep them, is what the authors argue action tokenization has been missing.