Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs

Say the Same, Act Differently. Diagnosing and Steering Text-Action Divergence in Reasoning VLAs. By Zihao Feng and five others, from the University of Southern California, the University of Arizona, and one other institution. Published on the third of November.

Reasoning vision-language-action models, reasoning VLAs for short, do two things at once. They control a physical system, and they produce a natural-language rationale that a supervisor can read to keep an eye on embodied safety. Earlier attacks on that reasoning corrupted the rationale itself, which leaves visible evidence in the text. The claim here is more unsettling. Visual perturbations can change the physical action while the generated rationale stays word-for-word identical. The text says the same thing, and the robot or the car does something else.

The authors look at the shared interface between text and action in two models: Alpamayo, which drives, and InstructVLA, which manipulates objects. Their diagnostic is called the text-neutral action-subspace, or TNAS. It finds that the direction to which the action is locally most sensitive is nearly orthogonal to the active text-loss gradients. The norm overlap is only one point eight seven percent in one model and five point two eight percent in the other. At equal perturbation norms on the hidden state, pushing along this text-neutral residual shifts actions between four point six four and forty-seven point eight times more than pushing along the text gradient. The reverse also holds: text-gradient perturbations cause between twenty-one point two and two hundred seventy-seven times more text drift. Steering visual perturbations along this direction achieves the same decoupling starting from the image itself.

Then the safety test. Across one hundred fifty-four driving scenes in Alpamayo annotated with obstacles, attacks guided by TNAS produce a new collision while the rationale stays word-for-word identical. That happens in fifty-two point six percent of scenes when the whole image is perturbed, and in thirty-five point one percent with only a localized patch on the forward camera. Unguided action attacks manage four point five five percent. Against a baseline that also tries to preserve the text, TNAS guidance lifts exact rationale preservation from eighty-three percent to eighty-nine point five percent, and cuts teacher-forced text drift by sixty-eight point four percent, at a comparable collision rate. In InstructVLA, the guidance cuts text drift by forty point five percent, keeps the exact response on eighty-five percent of examples, and still moves the action chunk by one point seven three units.

The evidence covers two models at one shared interface: Alpamayo in driving and InstructVLA in manipulation. The collision numbers come from the driving setting, from obstacle-annotated scenes, under full-image perturbations or a forward-camera patch. The manipulation results are measured as text drift, exact response preservation and action-chunk shift rather than as collisions. The orthogonality is described as a local property: the most action-sensitive direction near a given input, not a global statement about the whole model.

The consequence the authors draw is direct. An unchanged rationale cannot certify a safe action. If reading the text is how a supervisor monitors a reasoning VLA, then that monitor can be satisfied while the action has quietly changed underneath it. Their prescription follows from that: monitors for these models have to verify the predicted action channel directly, instead of trusting the explanation to reflect it.