RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting
Hejun Wang, Jinxi Li, Junwei Jiang, Shiwei Mao, Hu Cheng, Shouwang Huang, Bo Yang
The Hong Kong Polytechnic University
Published September 7, 2026 · on the feed September 9, 2026 · ▲ 10 · arXiv 2609.07414
The abstract
Image relighting is traditionally tackled via complex inverse rendering pipelines, which suffer from ill-posed optimization, or single-image generative models that ignore crucial multi-view cues necessary for understanding 3D geometry and material interactions. To address these limitations, we introduce a feed-forward generative Transformer for direct single- and multi-view image relighting that entirely bypasses explicit intrinsic property estimation. Adapted from a video foundation model, our architecture features a latent illumination module that dynamically injects target environment maps into spatial features via cross-attention. Furthermore, we employ permutation-invariant positional encodings to symmetrically process unordered multi-view inputs without sequential bias. To train this robust data-driven model, we construct the massive Laval Objaverse Dataset (LOD), comprising 90K objects and 39K unique illuminations. Extensive experiments demonstrate state-of-the-art visual quality, photorealistic relighting quality, and strong zero-shot generalization across single-view, multi-view, and novel-view relighting tasks.
The Brief
1:58A short podcast summary of the paper. Written and voiced by AI.
Transcript
RelightFormer. Feed-forward Generative Transformer for Multiview Object Relighting. By Hejun Wang and six others, from the Hong Kong Polytechnic University. Published on the seventh of September. Its code repository is public and real: about one hundred and ninety source files.
Relighting an image has traditionally gone one of two ways. Complex inverse rendering pipelines suffer from ill-posed optimisation. Single-image generative models ignore the multi-view cues that are crucial for understanding 3D geometry and how materials interact with light. One route is fragile in its optimisation, the other blind to what additional views could tell it.
RelightFormer is a feed-forward generative Transformer for relighting single- view and multi-view images directly, and it bypasses explicit estimation of intrinsic properties entirely. There is no intermediate step of recovering geometry or material properties and then rendering them again under new light. The input images and the target lighting go in, and the relit images come out. It is adapted from a video foundation model.
A latent illumination module injects the target environment map into spatial features dynamically, through cross-attention. Multi-view inputs arrive unordered, so the model uses permutation-invariant positional encodings to process them symmetrically, with no bias from the order they happen to come in. Any set of views is treated the same, however it is arranged. To train it, the authors built the Laval Objaverse Dataset, or LOD, a large dataset of ninety thousand objects and thirty-nine thousand unique illuminations.
Extensive experiments report state-of-the-art visual quality and photorealistic relighting, along with strong zero-shot generalisation across single-view, multi-view and novel-view relighting tasks. The generalisation is zero-shot, reported across all three of those task settings.
The approach removes the stage the older pipelines depended on. Instead of estimating geometry and materials and then re-rendering, the model learns to relight directly from data, using several views when it has them and treating those views without a fixed order.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Changing the light on an object sounds simple until you try to undo the old light.
You want to show an object under new lighting: same shape, same materials, just lit by a different environment. Film, games and augmented reality all need this.
The road ahead
- 1 · The problem ~6 min Showing an object under new light is hard: recovering its shape and materials is slow and ambiguous, and one-photo generators ignore the other views.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Give one generative Transformer the views and the target lighting, and let it output relit views directly, with no geometry or material estimation.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Adapt a video model, feed it lighting through cross-attention, give views camera-based positions, and generate the relit images directly.
- 6 · The experiment ~7 min You’ll guess: Trained on a rendered dataset, in which relighting settings did RelightFormer report strong zero-shot generalisation?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.