RepForcing: Representation-Space Initialization for Autoregressive Video Distillation

RepForcing. Representation-Space Initialization for Autoregressive Video Distillation. By Siming Fu and five others, from Joy Future Academy and Zhejiang University. Published on the fourteenth of October. Its code repository is worked out: about seventy source files, with a readme and a licence.

Turning a pretrained video diffusion model into a few-step autoregressive generator usually takes two ingredients: an initialization of the student, and distribution matching distillation, known as DMD. An intermediate autoregressive teacher can supply causally compatible targets for initializing from ODE trajectories. Compatibility alone, though, does not guarantee good generation when the sampling schedule is short. The authors analyze the posterior-mean predictions that direct regression in latent space tends to favour, and find empirically that causal ODE initialization reduces blur but still leaves fine detail unresolved.

Representation Forcing, or RepForcing, changes what the initialization is supervised against. Rather than regressing onto the trajectories of an intermediate teacher, it learns the few-step causal initialization through supervision in representation space. The diagnosis motivates the switch. Regressing directly in latent space pulls predictions towards a posterior mean, and a mean of many plausible videos is a blurry one. Comparing in a feature space instead changes what counts as a close match, and that is the lever the method pulls on fine detail.

The student is trained with teacher forcing. Its clean predictions and the training targets are compared through the normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos that a bidirectional model generated for training the autoregressive teacher, and it takes the place of both the teacher training and the causal ODE distillation. No additional causal ODE trajectories have to be generated. The asymmetric DMD procedure that follows is left unchanged.

At four sampling steps, RepForcing preserves generation quality better than initialization based on an intermediate teacher, measured before DMD. After DMD, the four-step chunk-wise model raises VBench Total from eighty-four point zero four to eighty-four point six six, a gain of zero point six two points.

VisionReward moves from six point three two six to ten point zero seven four, an increase of fifty-nine point two five percent. Both comparisons are against the reported baseline built on causal ODE initialization.

The change sits entirely in the first phase. Swap trajectory regression for feature comparison through a frozen transformer, drop the intermediate teacher and its ODE distillation, and keep the DMD stage as it was. The pipeline gets shorter, since two steps collapse into one that reuses videos already on hand, and the results suggest it also starts the final stage from a better place.