Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence

Rethinking Latent Visual Reasoning. Grounding Latent Reasoning in Visual Evidence. By Xi Xiao and twelve others, from the University of Alabama at Birmingham and Amazon AGI. Published on the twenty-eighth of September. Its code repository is public and real: about thirty source files.

Latent visual reasoning, LVR, lets multimodal large language models do their intermediate computation in continuous latent tokens, instead of spelling out every reasoning step in words. The cost is visibility. Unlike a textual chain of thought, latent reasoning cannot be read directly, which makes it hard to supervise what those latent tokens actually learn. With words, a wrong step can be spotted and corrected. With a vector, there is nothing to read, so there is no obvious place to put a correction either.

The authors begin by analyzing how latent tokens behave, and they find what they call a latent evidence-credit gap. Perturb an image in a way that changes the correct answer, and the latent tokens barely respond. Their hypothesis is that this comes from the lack of explicit supervision during GRPO training. A reward on the final answer alone gives too little guidance about which visual evidence to keep, or how to spread credit across the latent tokens. One judgment of the final answer has to stand in for everything the latent tokens did along the way.

ReaLVR is the response. It brings supervision grounded in visual evidence to the model's own latent trajectories, the ones it produces when running freely. The supervision is applied to what the model actually does at inference, not to some idealized path it never takes on its own.

Two contrasts do the work. Comparing correct answers with wrong answers the model generated itself shows where stronger supervision is needed. Comparing relevant visual evidence with mismatched evidence specifies what should be preserved. One contrast locates the problem, the other defines the target.

Across three model families, ReaLVR consistently outperforms the LVR baselines it was evaluated against. On Qwen2.5-VL-7B it reaches the highest five-task average, sixty-three point seven percent. The authors also report being the first to scale visual reasoning in latent space, with robust improvements continuing at frontier scales of up to two hundred thirty-five billion parameters.

Further analyses look inside the trained models. Latent-token positions become more sensitive to the question. Alignment with the relevant regions of the image gets stronger. And the most attended latent tokens show greater dependence on a fixed context. Each of these speaks to the gap the work began with, pointing to latent tokens more closely tied to the question being asked and to the parts of the image that bear on it.

Reasoning that cannot be read still has to be accountable to something. Here that something is the image itself: latent tokens are supervised against the evidence that matters, rather than being left to whatever a final-answer reward happens to shape.