Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation

Scene-Designer. Learning Design Intelligence for Agentic 3D Scene Generation. By Xuanyu Liu and seven others, from Zhejiang University and Universal Identification Technology, Hangzhou. Published on the sixteenth of October. Its code repository is worked out: about seventy-five source files, with a readme.

Turning natural language into complete, coherent and usable indoor 3D scenes takes what the authors call design intelligence: the ability both to design good scenes and to critique their quality. Existing approaches fall short in one of two ways. Either they rely on general-purpose models that lack domain-specific design experience, or they learn from static scenes, which show finished results rather than the process of making them. The first kind of model has breadth without expertise. The second sees what good scenes look like without any experience of building one, step by step, or of noticing what is wrong with one.

Scene-Designer is an agentic framework that learns design intelligence from two complementary kinds of experience, one of designing and one of critiquing. Each half of design intelligence gets its own supervision, and then the two are made to work together.

Two datasets supply them. Scene-OTA, thirty-five thousand examples, gives Observation, Thought, Action supervision for learning scene design and efficient error correction. Scene-OTC, fifteen thousand examples, gives Observation, Thought, Critic supervision for learning to judge scene quality across physical validity, functional correctness and layout coherence. On top of these sits Scene-GRPO, a Critic-guided GRPO framework that improves design decisions and cuts down on unnecessary trial-and-error corrections. The critique side is not only something the model learns. It is also used to steer the design side during training.

Extensive experiments show Scene-Designer performing best on physical validity, functional correctness and layout coherence. The learned Scene-Critic significantly outperforms general-purpose models at critiquing scenes. Its usefulness extends past training, too: Critic guidance at test time further improves scene generation. Additional analysis suggests a trend of mutual enhancement, with design and critique abilities each lifting the other.

Design and critique end up trained as two halves of one skill. The same critic that shapes the designer during training keeps helping when scenes are generated, and the analysis hints that getting better at one makes the model better at the other. Static scenes could teach what a finished room looks like. Experience of designing and of critiquing is what the authors offer for teaching how to get there.