Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning
Antoine Lorentz, Stéphane May, Valentine Bellet, Dawa Derksen, Bastien Nespoulous
Thales · Centre National d'Études Spatiales (CNES)
Published September 25, 2026 · on the feed September 28, 2026 · ▲ 14 · arXiv 2609.31199
The abstract
Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study diffusion models conditioned both on photogrammetric DSMs and Pléiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable Diffusion 3 architecture with a pruned text stream and a patch-wise normalization strategy, enabling stable training on LiDAR data and transfer from natural images to elevation maps. Experiments in French cities demonstrate that multimodal conditioning improves elevation accuracy, reducing Dense Urban RMSE from 6.00 to 3.45 m in the in-context cities and from 4.16 to 2.77 m in the held-out city of Bordeaux.
The Brief
1:43A short podcast summary of the paper. Written and voiced by AI.
Transcript
Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning. By Antoine Lorentz and four others, from Thales and the Centre National d'Études Spatiales. Published on the twenty-fifth of September. No code repository is linked to it.
Large-scale Digital Surface Models, or DSMs, can be produced cost-effectively from satellite images through stereo-photogrammetry. The resulting 3D maps, though, are often contaminated by noise, outliers and voids. Aerial LiDAR gives high-accuracy elevation measurements instead, but at a substantially higher cost. Cheap and flawed on one side. Accurate and expensive on the other.
The work studies diffusion models that refine vertically co-registered DSMs, conditioned on two inputs at once: the photogrammetric DSM itself, and Pléiades imagery. That multimodal conditioning is the point. The model sees both the rough elevation map and the optical image of the same ground.
The architecture is a modified Stable Diffusion 3, with a pruned text stream and a patch-wise normalization strategy. Together those changes enable stable training on LiDAR data, and transfer from natural images to elevation maps.
Experiments were run in French cities, and multimodal conditioning improves elevation accuracy. In the in-context cities, Dense Urban RMSE falls from six point zero zero to three point four five metres. In Bordeaux, a held-out city, it falls from four point one six to two point seven seven metres.
The result puts a model pretrained on natural images to work on a physical measurement problem. It takes the inexpensive satellite product and refines it, with LiDAR as the training target, and the improvement holds in a city that was held out.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Cheap maps from space, accurate maps from the air. You can rarely have both.
You map whole cities in 3D from satellite stereo pairs. It is cheap and covers huge areas, but the height maps come out noisy, with outliers and holes where matching failed.
The road ahead
- 1 · The problem ~6 min Satellite stereo makes cheap 3D city maps, but their heights are noisy and full of holes, while accurate laser scans are costly and rare.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Refine the noisy height map with a pretrained image generator, guided by the map itself and the satellite photo, after fitting elevations to image scale.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Prune the text stream, adapt the model to LiDAR with per-patch normalisation, add DSM and photo conditions, then refine stereo maps toward LiDAR.
- 6 · The experiment ~6 min You’ll guess: Did a diffusion model conditioned on both photogrammetric elevation maps and Pléiades imagery still reduce Dense Urban RMSE in a held-out city?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.