SUPRA-TEX: 3D Generation with Super-Aligned Texture

SUPRA-TEX. 3D Generation with Super-Aligned Texture. By Chen Yang and seven others, from Huawei, Shanghai Jiao Tong University, and one other institution. Published on the fifteenth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Single image 3D generation, as framed here, means dressing a given 3D mesh in relightable materials while faithfully keeping the visual details of a reference image. Existing methods fall short in one of two ways. Some fuse multi-view predictions that are inconsistent with one another, and introduce artifacts in the process. Others condition implicitly on image features, and struggle to preserve fine details. One route damages the picture while combining views, the other loses it on the way in.

SUPRA-Tex is built on a principle of decoupling three jobs: estimating the materials that are visible, aligning them to the surface, and completing the rest in 3D under guidance from the observation. That division of labour lets the 3D generator, which is trained on limited 3D data, spend its capacity on just two things: preserving the visible evidence, and completing the surfaces that were never observed.

First, an image editing model based on a multimodal large language model is trained to predict lighting-separated PBR maps in the reference view. Those maps are then projected onto the visible surface and encoded with the pretrained material VAE encoder. The result is a set of native observation latents that share a latent space with the generation target.

Generation itself is done by an interleaved flow transformer. It holds the observation latents fixed, as read-only anchors, and alternates between updates to the unobserved regions and updates to the full surface, until it has produced complete material latents. The authors call the outcome super-alignment with the reference: What You See Is What You Get fidelity on the visible regions, and coherent completion on the surfaces that were not observed.

Trained solely on public datasets, SUPRA-Tex reaches state-of-the-art performance in both geometry-conditioned texturing and end-to-end asset generation. On in-the-wild images, it matches and in some cases exceeds leading commercial systems.

Each stage does one job. The image model reads materials off the picture, projection pins them to the mesh, and the generator is left to fill in only what the camera never saw. Keeping the observed latents read-only is what holds the visible surface to the reference, and it is why the generator's limited training data goes toward the part of the problem nothing else can solve.