Reasoning with Image Generation
Nishad Singhi, Hector Garcia Rodriguez, Aditya Arora, Marcus Rohrbach, Anna Rohrbach
Technical University of Darmstadt · hessian.AI
Published September 14, 2026 · on the feed September 30, 2026 · ▲ 22 · arXiv 2609.16409
The abstract
Chain-of-thought reasoning has revolutionized natural language processing by enabling large language models (LLMs) to decompose problems into intermediate steps before answering. Yet confining reasoning to the textual domain presents limitations for tasks requiring direct manipulation of visual representations. Recent efforts augment multimodal LLMs with external visual expert tools such as depth estimation or object detection modules, but these remain fundamentally limited by their reliance on narrow, rigid operations that cannot flexibly generate or transform visual content. We propose ReImaGin, which leverages image generation models as a flexible visual reasoning mechanism for multimodal LLMs: unlike fixed-function tools, they accept natural language commands and can perform open-ended visual operations, like removing an occlusion or generating a floorplan from multiple disjoint views of a room. Across six diverse visual reasoning tasks including multi-view spatial reasoning and collision prediction, ReImaGin consistently outperforms both text-only reasoning and specialist vision-tool baselines, with gains of up to 25\%, demonstrating the advantage of flexible, generative visual reasoning.
The Brief
2:03A short podcast summary of the paper. Written and voiced by AI.
Transcript
Reasoning with Image Generation. By Nishad Singhi and four others, from the Technical University of Darmstadt and hessian AI. Published on the fourteenth of September. No code repository is linked to it.
Chain-of-thought reasoning changed natural language processing by letting large language models break a problem into intermediate steps before answering. But when the reasoning is confined to text, tasks that need direct manipulation of visual representations are left short. Recent work has given multimodal models external visual expert tools, such as depth estimation or object detection modules. Those tools are narrow and rigid. Each performs one fixed operation, and none of them can flexibly generate or transform visual content.
ReImaGin uses image generation models as the reasoning mechanism instead. Unlike a fixed-function tool, an image generator accepts natural language commands and can carry out open-ended visual operations. The multimodal model can ask it to remove an occlusion, for instance, or to produce a floorplan from several disjoint views of a room.
The difference lies in what an intermediate step is allowed to be. With a depth estimator, the model can only ever get back a depth map. With a generator, the intermediate step is whatever image the model can describe in words. The visual operation is chosen by the reasoning, not by the contents of a toolbox, and that is the flexibility the authors are after.
Evaluation covers six diverse visual reasoning tasks, among them multi-view spatial reasoning and collision prediction. On these, ReImaGin consistently outperforms two kinds of baseline: reasoning in text alone, and reasoning with specialist vision tools. The gains reach up to twenty-five percent.
One way to picture the contribution: a model that could only describe a scene in words, or point a single-purpose instrument at it, can now sketch. Taking away an object that blocks the view, or assembling a floorplan from scattered glimpses, becomes a step in the reasoning rather than a capability someone had to build in advance. The authors read the results as evidence for flexible, generative visual reasoning over fixed operations.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Some questions about a picture need a picture that isn't there yet. Start with that wall.
You are building a multimodal model that answers questions about pictures. It reasons well in words, but some questions need you to see something that isn't in the photo: what is behind an occluder, or a room from above.
The road ahead
- 1 · The problem ~6 min Some picture questions need you to see something the photo does not show, and both word-only reasoning and fixed vision modules fall short there.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min While reasoning, the model commands an image generator in plain language to make whatever picture it needs, then looks at it and carries on.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Equip the model with an image generator, reason in text, command the generator in words, then inspect its image to reach the answer.
- 6 · The experiment ~6 min You’ll guess: Across six visual reasoning tasks, how large were ReImaGin's biggest gains over text-only reasoning and specialist vision-tool baselines?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.