Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene

Mira-Scene. Pixel-Aligned Layouts for Generative 3D Scene. By Yang-Tian Sun and nine others, from the University of Hong Kong and VAST. Published on the twentieth of September. Its code repository is worked out: about one hundred and sixty source files, with a readme.

Generating a single 3D object from a single image now yields high-fidelity assets. Placing those objects correctly into a coherent scene is still open, and the central difficulty is how the layout is represented. Holistic methods fold placement into generating the whole scene at once, and lose object-level detail in the process. Compositional methods keep objects faithful by separating geometry from layout, but they usually represent layout as sparse pose variables with no bounds. Those are hard to learn, and with little scene-level supervision to go on, they generalize poorly.

Mira-Scene keeps the compositional approach but changes what is predicted. Instead of regressing a sparse pose, it recovers a dense, bounded correspondence. The core of this is the Canonical Coordinate Map, or CCM. It is a pixel-aligned field: for each visible pixel of an object, it gives the coordinate on the object's surface within that object's own bounded canonical space.

A CCM on its own says where each pixel sits on the object. Paired with a Point Cloud Map, a PCM in scene space that comes from monocular geometry estimation, it says where that same pixel sits in the scene. The two together induce dense correspondences from canonical space to scene space, and the object's transformation is recovered from them by robust geometric alignment.

The boundedness is what makes this learnable. Because the CCM lives in a bounded canonical space, it is a stable target to predict, and it can be trained on scalable object-level 3D data, with no need for scene-level layout annotations.

Mira-Scene also introduces a multimodal diffusion transformer that generates object geometry and CCMs jointly. It uses expert streams specific to each modality, with shared attention and shared positional encoding, to encourage the geometry and the layout to stay consistent with each other.

Experiments cover indoor, outdoor, synthetic and in-the-wild scenes. Mira-Scene substantially outperforms strong baselines in layout accuracy. Over SAM3D, the relative gains are thirty-nine point eight percent in 3D-IoU and sixteen point five percent in 2D-IoU. Those gains come while using only limited open-source training data, which is the regime the bounded, object-level target was built for: scene-level supervision stays scarce, and the model does not depend on it.

At heart, this is a change of target. A pose is a handful of unbounded numbers that a network must guess from little supervision. A canonical coordinate per pixel is dense and bounded, can be learned from objects alone, and hands the placement problem to geometric alignment, which can solve it from correspondences once they exist.