Learning to Think and Explore in 3D World

Learning to Think and Explore in 3D World. By Yijiang Li and two others, from the University of California San Diego and the Georgia Institute of Technology. Published on the fifteenth of October. No code repository is linked to it.

Vision-language models are often given a 3D scene in a form that hands them everything at once: multiple views, a video, or an explicit 3D representation. The question asked here is different. Can a vision-language model learn to think and actively explore in 3D space, the way a person does, seeing a single image at a time and choosing where to look next? That asks for two things the usual setup never requires. The model has to work out what it still needs to see, and it has to act in order to see it. A model that is shown every view in advance never faces either decision, because someone else has already made both of them on its behalf.

The authors keep the model's native visual encoding: one image, one at a time. The 3D representation stops being an input and becomes an environment. The model interacts with it through actions, such as moving left, and receives observations in the form of rendered images. Each rollout begins from an initial first-person view and a visual question. From there the model can reason, navigate and reorient its viewpoint, repeatedly, before it commits to a final answer. Thinking and moving are interleaved. A thought can prompt a move, the new image can prompt another thought, and the loop continues until the model decides it has seen enough to answer.

Training has two stages. An initial finetuning stage supervises the model on trajectories. Reinforcement learning follows, with rewards driven primarily by whether the final answer is correct. The crucial choice is what that reward leaves out. It gives no direct supervision over which viewpoints the model should visit, and none over which exploration actions it should take. The model has to discover an effective exploration policy purely from how much its interactions improve its performance. Looking around is rewarded only when it pays off in a better answer. Nobody tells the model where the useful views are. It has to find out, from the answers that its own choices of where to look make possible.

The resulting model is called ACT3D. Across six 3D benchmarks it reaches an average accuracy of fifty-three point two percent. That improves on its base Qwen model by twenty-two point zero percent, and surpasses the proprietary Gemini three point six Flash. It also outperforms existing open-source 3D methods, while using fifty to seventy percent fewer input frames. On unseen domains, it improves on its own supervised initialization by an average of five point nine percent.

Further analysis asks whether the exploration adapts to the question. On questions answerable from the initial view, the gain over the base model is zero point six percent. On questions that require extensive exploration, it rises to sixteen point three percent. In behaviour, ACT3D searches for targets that are occluded, avoids exploration it does not need, and gathers complementary viewpoints when a question calls for a spatial comparison.

The conclusion the authors draw reaches past accuracy. Outcome-based reinforcement learning, with a reward that looks only at the answer, teaches a vision-language model to reason over visual evidence and also to learn when and how to acquire that evidence. The shape of the gains fits that reading: almost nothing where looking around was unnecessary, and the most where it was essential.

Learning to Think and Explore in 3D World. By Yijiang Li and two others, from the University of California San Diego and the Georgia Institute of Technology. Published on the fifteenth of October. This is the long version: the whole paper rather than the abstract.

Most 3D vision-language work hands the model everything up front: a full scan, a point cloud, a set of views, or a video. The question is then a matter of reasoning over evidence that has already been gathered. A person standing in an unfamiliar room does not work like that. They see one view, and if the answer is behind them or around a corner, they turn and walk. The paper asks whether a vision-language model can learn the second habit: deciding what it still needs to see, and going to get it, as part of the reasoning itself.

The setup keeps the 3D scene outside the model. The model sees one rendered image at a time, starting from a fixed first-person view, along with a question. At each step it writes some reasoning and then either moves the camera or commits to an answer. The scene is hidden and never changes. Only the camera moves. Formally this is a partially observable decision process, and the appendix makes the core argument precise. Every new view can only narrow the set of scenes that agree with what has been seen. A model limited to the first view has an accuracy ceiling that no amount of capacity can lift whenever that view fits several scenes with different answers. An interactive policy can always do at least as well, because answering at once is one of its options. The catch the authors flag is that a single new view need not help. Gains come only from choosing informative ones.

Actions are relative to the camera. There are six translations and four rotations, each with a default step: one metre forward or back, zero point six metres sideways or vertically, thirty degrees for a turn. The model may ask for a larger step, up to eight units, which gives eighty navigation actions in all, plus the answer action. The camera stays level, is clamped to a box one metre larger than the scene, and has no collision checking, so it can pass through walls.

Training has two stages. First, GPT-5.4 generates reason, act, observe trajectories in this environment, filtered to keep only correct, valid and well-formed ones: five thousand seven hundred and twelve trajectories over eight hundred and thirty scenes. Supervised fine-tuning on these teaches the protocol, and the authors stress that it is an initialization, not a target exploration strategy. Second, reinforcement learning with group relative policy optimization samples eight rollouts per question, from the same start, and rewards each against its group. Since actions decide what is seen, the rollouts can gather quite different evidence, and the comparison credits better gathering.

The reward is mostly about the answer: an LLM judge's verdict of semantic equivalence, an exact-match term, and a penalty for a missing answer. The interesting part is the interaction term. It is paid only if the answer is correct. If the asked-about object was actually brought into view, the rollout gets the full interaction credit at once, even after one move. Otherwise credit accrues per move up to twelve. The stated intent is to pay for evidence, not for looking. Nothing in the reward names a viewpoint or a path.

Across six benchmarks, the model, called Act3D and built on Qwen3.5-9B, averages fifty-three point two percent, twenty-two points above its base and one point above Gemini 3.6 Flash under the same protocol. It starts from a single image, at fifty to seventy percent fewer input frames than existing open-source 3D methods.

The comparison that most directly tests the thesis is the stratified one. Validation questions are grouped by how many moves a geometric estimate says it takes to reach the answer object, a measure independent of what the model actually does. The combined gain over the base model rises from zero point six percent where the answer is already in view to sixteen point three percent for the hardest group, where reinforcement learning alone adds ten point nine over fine-tuning. During training, interactions per episode climb from about three to about eight.

The ablations carry the design. Relative actions beat absolute camera poses, forty-one point four percent against thirty-six point seven. Rendered views beat retrieving the nearest real photograph at every one of thirteen checkpoints, by ten point six points on average, because retrieved photos can sit well away from the requested pose. And reinforcement learning with no fine-tuning does learn to interact, but slowly: about one move per episode at first, two after four hundred and seventy steps.

The paper has no limitations section. Its own tables show a few things the headline hides. The margin over Gemini is one point. On ScanQA the passive Ross3D still scores higher, and the authors reproduced Ross3D's published numbers to make that comparison fair. Reinforcement learning on mixed data improved five of six benchmarks, not all. The no fine-tuning comparison uses a different backbone and training budget, which the authors acknowledge. The scenes are reconstructed indoor scans with a camera that ignores collisions, so this is camera control, not embodied navigation. And the training curves report a single draw, with no error bars on the main table.

The idea to carry away is that evidence gathering can be learned from the answer alone. Give a model one view, a way to move, and a reward for being right, and it learns to search when the target is hidden, to stop early when it is visible, and to seek a second angle when comparing objects. The gain is largest exactly where the first view is least enough.