SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes?

SpatialScout. Can VLM Agents Actively Explore and Map 3D Scenes?. By Zhaoxi Zhang and nine others, from Peking University and Tianjin University. Published on the nineteenth of October. Its code repository is worked out: about three hundred source files, with a readme.

Spatial intelligence asks an agent to go and get visual evidence for itself, and to integrate what it sees into a coherent representation of the three-dimensional world. Spatial question answering, the usual test, offers limited evidence on one crucial point: whether a model can build an accurate metric map through its own exploration. Answering a question about a scene is not the same as mapping it.

SpatialScout evaluates vision-language model agents on active, object-level 3D mapping. The agent is given an inventory of objects, RGB observations, camera calibration and camera poses. It controls a camera to explore each scene, and has to estimate the centres of the listed objects in one shared coordinate frame. Doing that well means coordinating several skills at once: choosing viewpoints, associating the same object across views, estimating geometry, and revising the map, all within a bounded budget of interactions.

The benchmark comprises eighty diverse indoor scenes containing three thousand one hundred and forty target objects. Scoring measures two things jointly: how accurately objects are localized, and how many of the targets are covered. It does this through overlap scores at the scale of each object, and those scores penalize missing predictions, so an agent cannot score well by mapping only the few objects it is sure of.

Frontier VLMs show substantial limitations. The strongest model evaluated, Claude Opus 5.5, achieves a mean overlap score of forty-one point seven one percent.

Looking at thresholds tells the rest. Forty-seven point two three percent of its targets exceed an intersection-over-union, or IoU, of zero point five. Only three point two five percent exceed zero point nine. The authors read this as a substantial gap between recovering a coarse layout of objects and constructing a complete, precise metric map.

The benchmark turns a passive question into an active task. Instead of asking a model about a scene it has been shown, it hands over the camera and asks for a map. On that task, even the best model evaluated places a little under half its targets roughly and very few of them precisely, and SpatialScout is offered as a quantitative testbed for agents meant to close that gap through active perception.