FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation
Yijing Lin, Zhendong Mao, Mengqi Huang, Yuxin Wang, Yongdong Zhang
University of Science and Technology of China · FrameX.AI
Published October 27, 2026 · on the feed November 2, 2026 · ▲ 34 · arXiv 2610.36545
The abstract
Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.
The Brief
2:12A short podcast summary of the paper. Written and voiced by AI.
Transcript
FrameForge. Harnessing Priors of Video Foundation Models for Unified Image Generation. By Yijing Lin and four others, from the University of Science and Technology of China and FrameX AI. Published on the twenty-seventh of October.
Unified conditional image generation asks one model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models offer a promising foundation, because they encode visual transformations across time. Existing approaches, however, typically assign the condition and the target to fixed temporal positions. That treats every task as the same kind of step, however small or large the change it asks for.
The authors observe that the relationship between condition and target varies substantially from one generation task to another, while video models naturally encode transformations at different temporal scales. The two observations fit together. If a video model already holds changes that unfold over short spans and over long ones, each task could draw on the span that suits it. That motivates FrameForge, which models conditional generation through virtual temporal relations that adapt to the input. It captures two complementary aspects of those relations: how far, and how.
Mixture of Temporal Experts, or MoTE, handles how far. It models the distance between condition and target by dynamically selecting among temporal RoPE experts, each with a different virtual interval. Chain of Temporal Reasoning, or CoTR, handles how. It models the transformation of condition into target through a latent intermediate state, and a reasoning-bottleneck attention mask encourages information to propagate through that pathway, rather than letting the target be produced without passing through it.
Experiments on DreamBench, COCO2017, and GEdit-Bench show FrameForge achieving superior performance among the compared methods in three settings: subject-driven generation, canny-to-image generation, and image editing. The three ask for different relationships between what is kept and what is changed, which is the variety the input-adaptive design was built to handle.
Time, in this framing, becomes a dial rather than a slot. Where earlier approaches put the condition and target at fixed positions, FrameForge lets each task decide how far apart they sit, and lets the route between them pass through a state in the middle. What a video model learned about change over time is put to work on single images.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
One model, many kinds of image change, and a video model that knows about time. Where to start?
You want one model to handle image editing, subject-driven generation and turning edge maps into pictures. A pretrained video model looks promising, since it already knows how scenes change over time.
The road ahead
- 1 · The problem ~7 min A video model knows how scenes change, but fixing the input and target at set frame positions treats every task as the same size of change.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Treat input and target as video frames whose time gap adapts to each input, and route the change through a middle latent.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Encode images as video frames, route them through a chosen frame gap and a middle latent, then generate the target with the frozen video model.
- 6 · The experiment ~6 min You’ll guess: Across subject-driven generation, canny-to-image generation and image editing, how did the single FrameForge model fare against the compared methods?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.