S-Agent 2.0: Spatial Reasoning with a General Coding Agent

Rui Gong, Ruoyu Wang, Yalun Dai, Hao Li, Sidi Yang, Qihao Zhao, Hongyu Li, Zeyang Guan and 4 others

Nanyang Technological University · University of Hong Kong · Ropedia

Published October 20, 2026 · on the feed October 21, 2026 · ▲ 106 · arXiv 2610.28354

AgentsCode & softwareReasoning & math Full code arXiv ↗Hugging Face ↗

The abstract

Spatial reasoning asks a model to work out where objects are in 3D space, how they relate to one another, and how they move. Agentic VLMs have recently brought large improvements on such tasks by letting the model actively gather evidence with 3D perception tools instead of answering in a single pass. Even so, on many questions the tools return the right evidence and the model still does not arrive at the correct answer: the gap between model and tools, where raw readings become usable conclusions, is left unfilled. We present S-Agent 2.0, a spatial reasoning agent that equips an off-the-shelf general coding agent with a command-line perception toolkit, and we design the interface around one principle: help the VLM close the evidence loop: acquire scene evidence, remember it in a persistent scene memory, and interpret each reading in plain spatial language. This design keeps the full flexibility of a coding agent, and every reading it acts on stays grounded in the tools' geometry. On MMSI-Bench and ReVSI, the toolkit improves a strong open-source model, Qwen3.8-27B, by +13.9 and +20.5 points over the same agent without tools, and a strong closed-source model, Gemini-3.8-Flash, by +11.2 and +9.6 points. With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7. Both tool gains shrink as the backbone grows more capable, but SpatialClaw's shrinks far more on ReVSI, from +13.2 to +2.7, while ours stays at +9.6. S-Agent 2.0 also works across different general coding agent harnesses, such as Codex and Claude Code.

The Brief

3:10

A short podcast summary of the paper. Written and voiced by AI.

Transcript

S-Agent 2.0. Spatial Reasoning with a General Coding Agent. By Rui Gong and eleven others, from Nanyang Technological University, the University of Hong Kong, and one other institution. Published on the twentieth of October. Its code repository is worked out: about two hundred and fifty source files, with a readme and a licence.

Spatial reasoning asks a model where objects sit in three-dimensional space, how they relate to each other, and how they move. Agentic VLMs have recently improved a great deal on these tasks, because instead of answering in a single pass they actively gather evidence with 3D perception tools. But the authors point to a stubborn failure. On many questions the tools return the right evidence, and the model still reaches the wrong answer. Between the model and its tools sits a gap, the place where raw readings should become usable conclusions, and nothing currently fills it.

S-Agent 2.0 is a spatial reasoning agent, and it is not built from scratch. It takes an off-the-shelf, general coding agent and equips it with a command-line perception toolkit. The effort goes into the interface between the two, which is designed around a single principle: help the VLM close the evidence loop. Getting a reading is only the start of that loop; the reading also has to be kept, and it has to be understood.

That loop has three steps. First, acquire evidence about the scene through the tools. Second, remember it, in a persistent scene memory, so that a reading taken earlier is still there when a later step needs it. Third, interpret each reading in plain spatial language, so the model receives a statement about where things are rather than a bare number it must decode. Built this way, the agent keeps the full flexibility of a coding agent, free to compose its own steps, while every reading it acts on stays grounded in the geometry the tools computed.

The tests use MMSI-Bench and ReVSI. Compared with the same agent running without tools, the toolkit lifts a strong open-source model, Qwen3.8-27B, by thirteen point nine points on MMSI-Bench and twenty point five points on ReVSI. A strong closed-source model, Gemini-3.8-Flash, gains eleven point two and nine point six points on those same two benchmarks.

On the open backbone, S-Agent 2.0 reaches sixty-six point five on MMSI-Bench and eighty-two point three on ReVSI. The specialised system SpatialClaw, on the same backbone, scores sixty-two point six and seventy-four point five.

With Gemini-3.8-Flash as the backbone, S-Agent 2.0 reaches seventy-four point one on MMSI-Bench and ninety point five on ReVSI. SpatialClaw, on that backbone, gets sixty-seven and seventy-nine point seven. So the lead holds on both benchmarks with both backbones.

Both systems gain less from tools as the backbone becomes more capable. But on ReVSI, SpatialClaw's gain shrinks far more, falling from thirteen point two points to two point seven, while the gain for S-Agent 2.0 holds at nine point six. The agent also works across different general coding agent harnesses, including Codex and Claude Code.

What sets this apart from simply adding more tools is where the effort goes. The perception was already returning the right numbers; S-Agent 2.0 spends its design on the step after, turning a reading into something the model can remember and reason with.

Paper Quest · 7 chapters · about 41 min

Play this paper

Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.

Chapter 1 🧩 · The problem

The tools see the room correctly. The answers still come out wrong. Start there.

You run an agent that answers spatial questions about photos of a room: where things are, how they relate, how they move. It can call 3D perception tools that measure the scene.

The road ahead

  1. 1 · The problem ~7 min The 3D tools measure the room correctly, yet the agent still answers spatial questions wrong, because the readings never turn into usable conclusions.
  2. 2 · What came before ~3 min
  3. 3 · The key idea ~9 min Keep a stock coding agent and build its 3D tools so that every reading is acquired, remembered, and interpreted, closing the evidence loop.
  4. 4 · Choose your own research ~3 min
  5. 5 · Build the method ~8 min A stock coding agent gets perception as shell commands, then acquires evidence, stores it in a scene memory, and reads it out in spatial language.
  6. 6 · The experiment ~6 min You’ll guess: As the backbone moves from Qwen3.8-27B to the stronger Gemini-3.8-Flash, what happens to SpatialClaw's tool gain on ReVSI compared with S-Agent 2.0's?
  7. 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Join free to play the whole quest ›

Free, with Google or an email address. Your stars and progress are kept.

KipQuest: the day’s AI research, as podcasts and quests. All papers Android app Privacy Contact Episodes are written and voiced by AI from each paper. Summaries can contain mistakes: the paper is the source.