History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

History Forcing. Distilling History-Dependent Continuations for Long-Horizon Video World Models. By Hangyi Kuang and ten others, from Zhejiang University, Microsoft Research, and two other institutions. Published on the second of November.

Few-step autoregressive video diffusion makes generation efficient. The video is produced piece by piece, each piece in only a handful of steps. The weak point is consistency over a long rollout, and it shows most clearly when a subject or a scene goes out of sight and later comes back. Distillation is how these fast students are trained from a teacher, and existing distillation methods have improved both fidelity and robustness. What they have not fully done is use richer historical information to constrain what the student predicts right now.

History Forcing is a unified on-policy distillation framework aimed squarely at that gap: it strengthens the historical supervision a few-step autoregressive generator receives. The key move is to isolate what history contributes. The teacher is asked for a prediction twice, once with an informative historical context and once without it. The residual between those two predictions is, in effect, the part of the teacher's answer that history is responsible for, and that residual decouples the historical supervision from everything else.

Where does an informative context come from? The abstract names two sources. One is earlier generated chunks, which carry less accumulated drift. The other is a fixed-view teacher rollout that keeps the scene in view while the student looks away. Either way, the residual is weighted and added to the score or velocity prediction used for distillation. That produces a corrected target. The student is then aligned with the corrected target through distillation, and in that way long-range historical constraints are built into autoregressive generation.

Among models of one point three billion parameters, the method reports state-of-the-art performance. For fifteen-second video, its VBench quality score is eighty-five point zero seven. For sixty-second generation, its VBench-Long aggregate score is seventy-nine point eight two.

The out-of-sight tests ask whether the world evolves plausibly while unseen, and whether it is still consistent when the camera returns. On PlayWorld, Evolution Quality rises from two point three five to two point seven. On MemoBench, Object Consistency rises from fifty-eight point four three to sixty-two point one eight.

The appeal of the approach is how targeted it is. Rather than handing the student more history and hoping it learns to use it, History Forcing measures what history changes in the teacher's prediction and teaches exactly that difference. A scene that leaves the frame is meant to be remembered, not reinvented, when the camera swings back.