SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution

SceneMosaic. Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution. By Xingjian Ran and five others, from the University of Hong Kong and the University of Electronic Science and Technology of China. Published on the fourth of September. Its code repository is worked out: about one hundred and twenty source files, with a readme and a licence.

Indoor scenes that are diverse and ready for simulation matter for interactive entertainment and for embodied AI, and generating them at scale is still hard. Two paradigms dominate, and each falls short in a different way. Agentic text- to-3D pipelines built on vision-language models, or VLMs, can produce high- fidelity scenes, but they pay for it with costly rounds of iterative object placement and refinement. Parametric image-to-3D scene models work efficiently, drawing on strong priors learned from 2D images, but their scenes are often imprecise and physically invalid: objects in the wrong place, or arrangements a simulator could not accept.

There is a further problem the two share, and the authors call it more important. Both struggle to produce diverse scenes from a single input. Real scenes change dynamically, and a generator that returns essentially one answer per prompt cannot reflect that.

SceneMosaic is a framework that combines the merits of both paradigms. The initial candidate comes from the learned image-based prior, which is fast and gives a strong starting point. That result is then evolved by VLM agents. Starting from the prior keeps the process efficient, and evolving it through agents is what secures physical validity. Neither paradigm has to carry the whole job alone.

Inside the evolution process, SceneMosaic exploits the locality of natural scenes. A scene is decomposed into independent local units, and evolution happens separately within each unit. The global scene is then composed from those units via a Cartesian product. That composition step is where diversity enters. A Cartesian product takes every combination of the parts, so separately evolved local units can be put together into many different complete scenes from the same starting point.

Evaluation is on SceneEval-100. SceneMosaic matches the strongest agentic baseline in semantic layout quality, and does so with a twenty-four times speedup. It substantially reduces physical violations, and it receives the highest ratings from human evaluators. So all three targets appear together: quality on par with the agentic approach, efficiency far beyond it, and physical validity improved.

Each paradigm's weakness is covered by the other's strength. The image prior supplies a quick first draft, the agents correct it into something physically valid, and the observation that natural scenes are local turns one draft into many variations. The code is publicly available.