Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips

Frechet Forcing. Training Long-Horizon Streaming World Models from Short Clips. By Yian Zhao and five others. Published on the nineteenth of October. Its code repository is worked out: about fifty-five source files, with a readme.

A streaming video world model generates the future chunk by chunk, conditioned on its own past and on a stream of actions. Trained on short clips, these models drift once a rollout runs beyond the training horizon. Prediction errors re-enter the context, and from there they compound.

A common remedy is to train the model on its own rollouts through distribution matching distillation, or DMD. That needs a teacher, and for world models conditioned on actions the teacher becomes the bottleneck. An external video model is agnostic to the actions, so the actions are invisible to it. The model's own pretrained network knows the actions, but it drifts in exactly the same way as the student it is meant to correct.

The way out starts from an observation. A long rollout does not need to match a teacher. It needs to remain a realistic continuation of its own scene, and to follow the actions it was commanded. Those requirements can be checked without a teacher. Realism and scene fidelity can be measured against statistics of real data, and those statistics stay valid at any horizon. Action following is already preserved by the model's own conditional generation loss. Fréchet Forcing builds on this, fine-tuning a world model together with the multi-step sampler it is actually deployed with, under three constraints.

The first is generative anchoring. It applies the flow-matching loss after a history that the model generated itself, ending on real data.

The second is context fidelity. This is a Fréchet distance, or FD, computed on latent statistics of the free rollout, centred on each clip's own ground truth. It keeps every rollout faithful to the scene it started in.

The third is reality matching, an FD to real frames measured in a frozen perceptual space, which keeps the generated frames realistic.

Read together, the three divide the work cleanly. Anchoring holds on to action following through the generation loss, after the model has had to cope with a history of its own making. Context fidelity asks whether the rollout is still the scene it began as. Reality matching asks whether the frames still look real.

All three targets are derived from short real clips. At no point is a rollout compared against a long recorded future. So training needs no teacher, no online critic, and no long videos. The thing that was missing for long horizons, a reference that is itself long, is simply never required.

The test case is MiniWorld-1B, a model pretrained on clips of two hundred and fifty-three frames. It is fine-tuned on clips of three hundred and eighty-one frames, for at most thirty-six GPU-hours. The resulting rollouts remain stable beyond four thousand frames, and they continue to follow their actions. That holds in two settings: robot manipulation, and exploration where the camera is under control. The gap between those numbers is the point: short training clips, and rollouts that run roughly an order of magnitude longer without coming apart.

The usual way to fix drift is to find something that can see the long future and copy it. Fréchet Forcing declines that route. Realism and fidelity to the scene are judged against real-data statistics that do not care how long the rollout has run, and the action signal comes from the model's own loss. A model fine-tuned only on short clips then holds together for thousands of frames, with no teacher anywhere in the loop.