SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning

SpatialSpeak. QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning. By Yang Cao and six others, from the Hong Kong University of Science and Technology, Huawei Noah's Ark Lab, and one other institution. Published on the twenty-seventh of September. Its code repository is a placeholder: a readme, and nothing to run yet.

Vision-language models can reason better about space across multiple views when they have geometric priors to lean on. The trouble is how they are usually trained. If a model is supervised only on final answers, nothing directly teaches it the intermediate geometric estimates, or how to use those estimates to derive a quantitative spatial answer. The estimates stay implicit and unchecked. Training rewards the final number, while the geometry that ought to produce that number is never itself taught.

The hypothesis here is about order. Spatial chain-of-thought supervision, the authors propose, works better if the model has first learned two complementary things through multi-view reconstruction: fine local geometry, and the global context of the scene. SpatialSpeak is a two-stage framework built on that hypothesis. The first stage teaches reconstruction. The second teaches reasoning that uses it.

Stage I is called QA-Native Reconstruction Pretraining, or QA-RP. It combines two kinds of query. Marked-point 3D queries target fine-grained local geometry. Object-center queries target global scene context across views. Crucially, both are posed as ordinary text question answering, so geometric estimation and the reasoning that follows it share one autoregressive output interface. The model states its geometry in the same channel it will later reason in.

Stage II is spatial chain-of-thought with Visual Compensation, or CoT-VC. Here the model is trained to state the geometric estimates relevant to a question, then use them to reach the answer. Alongside that, a reliability assessment and a visual compensation step support refining the answer when it needs it.

On ReVSI, the reconstruction pretraining more than doubles what the reasoning stage contributes: the gain from CoT-VC rises from two point six points to six point nine points once QA-RP comes first. Ablations show that the local and the global reconstruction supervision each help. Overall, SpatialSpeak reports state-of-the-art results on ReVSI, VSI-Bench and SPAR-Bench. Its ReVSI score of sixty-two point eight is eight point seven points above the strongest baseline it was compared against.

A useful way to hear this result is as a lesson about sequencing. Asking a model to reason step by step through geometry pays off far more when it has already practised saying that geometry out loud, in the same words it will reason with. The pretraining is what gives the chain of thought something solid to stand on.