Spatial Harness: Grounding Spatial Reasoning in Executable World States with Physics Engines

Spatial Harness. Grounding Spatial Reasoning in Executable World States with Physics Engines. By Chentao Cao and four others, from Hong Kong Baptist University and ByteDance Seed. Published on the eighth of November.

Spatial reasoning asks two things of a model. First, understand how the objects and the observers in a scene are arranged. Then, work out how those relations change when the viewpoint shifts or something moves. Vision-language models usually do both inside their own reasoning: they form an implicit reading of the scene from partial views, then imagine what a transformation would do to it. Either step can fail. Misread the scene, and the answer goes wrong from the start. Read it correctly, and there is still no guarantee that the imagined transformation respects spatial constraints.

Spatial Harness takes both steps out of the model's head. Scenes become explicit world states that can be executed, and the reasoning is grounded in feedback from a physics engine rather than in imagination.

Two roles divide the work. A Perceiver builds the state. It takes scene constraints extracted from the question, combines them with specialized visual estimates, and places objects and cameras together in one shared coordinate frame. That state is then instantiated in a physics engine, and a Reasoner takes over. It executes viewpoint changes inside the engine, and draws on computed measurements and rendered views to guide its reasoning.

Evaluation covers MindCube, SAT, MMSI-Bench and SpatialRGPT-Bench, with both open-weight and closed-weight vision-language models. On MindCube, Spatial Harness lifts Qwen3.5-27B from fifty-two point seven percent to seventy-one point one four percent. That surpasses pySpatial by seven point five two percentage points.

A controlled comparison isolates the reasoning half. When both approaches are handed identical annotated object and camera poses, Spatial Harness reaches ninety-eight point two five percent. Model-internal reasoning, given the same poses, reaches seventy-one point eight one percent.

That last pair of figures separates the two failure points named at the start. With perception held fixed and correct, the remaining gap belongs to the imagined transformation, and executing it in an engine closes most of the distance. A model that only has to read a scene, and can hand the geometry to a simulator, is being asked to do less of what it was getting wrong.