Soft Spatial Reasoning

Soft Spatial Reasoning. By Rafi Ibn Sultan and six others, from Wayne State University, Henry Ford Health, and one other institution. Published on the thirtieth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

Large vision-language models usually reason about space through chain of thought, writing out intermediate steps as an autoregressive sequence of discrete language tokens. The authors call this hard thinking, and its cost is that every step must commit to exactly one token, even when the correct spatial interpretation is still uncertain. They name the result premature discretization. Commit to the wrong token early, and the error can propagate through everything that follows. Left or right, in front or behind: once the sequence has said one, the alternatives are gone.

Soft Spatial Reasoning is a post-training framework that brings soft thinking to spatial tasks. At each intermediate step, instead of selecting a single token, the model forms a continuous soft state by mixing token embeddings. Several candidate continuations can then all influence the next step, rather than one winning outright and the rest being discarded.

There is a catch, and the paper builds around it. The right amount of softness is not the same at every step. Keeping several candidates alive can preserve a spatial interpretation that turns out to be useful. But if those candidates imply spatial relations that conflict with each other, mixing them can interfere with the reasoning that comes after. Sometimes it helps to stay undecided, and sometimes it does harm.

The component at the centre of the framework is AdaptSoft, a controller that sets the degree of softness at each reasoning step. It decides from two inputs: the current hidden state, and the model's predictive uncertainty. Training such a controller raises its own difficulty, because there is no supervision telling it what the intermediate reasoning should look like. The authors answer this with a gradient-alignment learning objective, which supplies a learning signal specific to each step for controlling softness, without needing intermediate reasoning supervision at all.

On a diverse set of spatial benchmarks, Soft Spatial Reasoning outperforms two kinds of chain-of-thought baseline built on the same backbone: the usual hard version, and a soft version with fixed softness. It also outperforms a range of existing large vision-language models. The abstract gives no specific numbers for these comparisons.

The comparison against fixed softness is the one that carries the argument. Beating hard thinking shows that keeping options open can help. Beating a fixed blend shows something narrower and more useful: that how open to keep them should be decided step by step, by looking at where the model is and how unsure it is.