SpatialBrain: Spatial Intelligence Growing from Experience

SpatialBrain. Spatial Intelligence Growing from Experience. By Yifan Zhou and seven others, from Shanghai Jiao Tong University and Zhejiang University. Published on the twenty-first of October. No code repository is linked to it.

Having geometric tools is not the same as reasoning spatially. A reasoner has to decide when looking directly is enough, and when it should instead reconstruct, measure, or change reference frames. Those decisions are the hard part, and they are what this work tries to learn from experience.

SpatialBrain is a framework that retains experience about those decisions while its vision-language backbone stays fixed. Its central distinction is between two kinds of knowledge. One is evidence about a scene. The other is experience about how to reason with that evidence. The framework keeps them apart. Evidence belongs to the scene in front of it and is gathered fresh. Experience is about method, the judgement of when to look, when to reconstruct, and when to measure, and that is the part meant to carry over from task to task.

Three parts carry the design. 3DFusionMemory fuses observations of the current scene into spatial relations that can be queried. A Spatial Skill Library provides reusable tool procedures. And Spatial Cognition associates task conditions with experience about when and how to use the memory and the skills.

Experience accumulates like this. Completed trajectories are abstracted into process records, and those records are consolidated between blocks of tasks. Later decisions can then draw on earlier execution, without carrying over scene coordinates or reference answers. What transfers is the know-how, not the particulars of any one scene.

Across five spatial-reasoning benchmarks, SpatialBrain improves on direct answers from the same backbone on four, and ties on the fifth. Component comparisons, experience-coverage curves, and traced resource decisions are used to characterise how the framework builds up and uses its memory, skills, and experience.

The comparison that matters is against the very same backbone answering directly. Nothing in the model's weights changes. What changes is a stored record of how earlier problems were approached, kept separate from the facts of any particular scene.