GeoVerse: World-Consistent Novel View Synthesis in Geometric Latent Space
Kerui Ren, Tao Lu, Linning Xu, Changjian Jiang, Mu Huang, Chunhua Shen, Mulin Yu, Bo Dai
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong · The University of Hong Kong · Fudan University · Zhejiang University
Published September 28, 2026 · on the feed September 29, 2026 · ▲ 8 · arXiv 2609.35734
The abstract
Novel view synthesis from sparse images must reconcile faithful reconstruction of observed regions with plausible completion of unseen content, while maintaining world consistency across viewpoints. Existing geometry-based methods preserve observed scene structure but often struggle to complete unseen regions, whereas video generative models offer rich appearance priors but accumulate inconsistencies during sequential view generation. We propose GeoVerse, a framework that synthesizes world-consistent novel views by performing generation within the geometric latent space of a pretrained 3D foundation model and injecting appearance priors from a video generative model. Specifically, GeoVerse extracts multilevel features from Wan2.2 VACE and injects them into the geometric latent diffusion model via a ControlNet-style adapter, incorporating video-learned appearance priors to enhance structural completion. To enforce cross-view coherence, a global spatial memory continuously aggregates observed and synthesized content, reprojecting target-aligned guidance to anchor subsequent predictions to a shared scene representation. Extensive experiments across diverse datasets demonstrate improved visual quality and geometric consistency, with a 2.23 dB higher PSNR on DL3DV and 32.4% lower ATE on Mip-NeRF360 compared to GLD.
The Brief
2:10A short podcast summary of the paper. Written and voiced by AI.
Transcript
GeoVerse. World-Consistent Novel View Synthesis in Geometric Latent Space. By Kerui Ren and seven others, from Shanghai Jiao Tong University, the Shanghai Artificial Intelligence Laboratory, and four other institutions. Published on the twenty-eighth of September. Its code repository is a placeholder: a readme, and nothing to run yet.
Synthesizing new views of a scene from a handful of images asks for two things that pull apart. Regions that were observed should be reconstructed faithfully. Regions that were never seen should be completed plausibly. And the result has to stay consistent as the viewpoint moves. Geometry-based methods preserve the observed structure but often struggle to fill in what is missing. Video generative models bring rich appearance priors, but they accumulate inconsistencies as they generate one view after another.
GeoVerse takes the strengths of both. Generation happens inside the geometric latent space of a pretrained 3D foundation model, which keeps the structure anchored, while appearance priors are injected from a video generative model to help with completion.
Concretely, GeoVerse extracts features at several levels from Wan2.2 VACE and injects them into the geometric latent diffusion model through a ControlNet-style adapter. Those video-learned appearance priors are what strengthen the completion of structure. For coherence across views, a global spatial memory keeps aggregating content, both observed and synthesized. It reprojects guidance aligned with the target view, so each new prediction is anchored to one shared scene representation rather than drifting on its own.
Across diverse datasets, the experiments show gains in visual quality and in geometric consistency. Compared with GLD, GeoVerse reaches a PSNR two point two three decibels higher on DL3DV, and an ATE thirty-two point four percent lower on Mip-NeRF360. The two figures line up with the two claims made for the method, the first speaking to visual quality and the second to geometric consistency, on two different datasets.
Each half of the design answers one half of the problem. The geometric latent space keeps what was seen where it belongs. The video prior supplies what was not seen. The spatial memory ties every new view back to the same world, which is the failure sequential video generation had on its own.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A few photos, a whole world to show. Every quest starts with a wall.
You have a handful of photos of a scene and must render new views of it. Parts the camera saw should look exactly right; parts it never saw must be filled in plausibly.
The road ahead
- 1 · The problem ~7 min New views from a few photos must copy what was seen exactly and invent what was not, while staying one consistent world.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Generate new views inside a 3D model's geometric feature space, add video-learned appearance knowledge, and anchor every step to a shared point memory.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Pull features from the video model, feed them through an adapter, fuse every view into a point memory, then project it into the next view.
- 6 · The experiment ~6 min You’ll guess: Compared with GLD, how much lower was GeoVerse's ATE on Mip-NeRF360?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.