SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

SolarWM. Open Data and Scalable Training for Long-Horizon Video World Models. By Junchao Huang and seventeen others, from the Chinese University of Hong Kong, Shenzhen, SLAI, and seven other institutions. Published on the second of September. Its code repository is worked out: about two hundred source files, with a readme and a licence.

What now exists is a fully open foundation for building interactive video world models, covering the whole path from data preparation through to long-horizon inference. Data, pipeline, recipes, weights and framework are all released, which is a wider release than the usual weights-and-a-paper.

The problem it addresses is coupling. Training across heterogeneous sources and video backbones is hard because datasets differ in temporal scale, camera geometry, visual quality, motion and captioning style, while video generators use distinct representations and architectures. Naive data mixing and model-specific implementations then produce inconsistent supervision and results nobody can reproduce or compare. Two components separate those concerns: a reconfigurable multi-source data engine, and a backbone-native adaptation framework.

The engine converts one point four three million canonical clips from ten datasets into a unified frame-aligned contract. What that contract carries is the interesting part: visual observations, metric camera geometry, captions, quality metadata, the selection decisions that were made, and provenance. Recording the selection decisions and the provenance is what makes a mixture auditable rather than merely reproducible, and the engine decouples source processing from mixture construction so the two can vary independently.

Under shared camera-conditioning, training and inference interfaces, the authors instantiate four models spanning five to thirty-three billion parameters on three different backbones, preserving each one's native representations and objectives rather than forcing them into a common mould. A three-stage recipe combines bidirectional adaptation, teacher-forced autoregressive initialization, and distribution matching distillation. The resulting causal models support real-time interaction over rollouts lasting minutes to hours, despite having been trained on sequences of only five seconds.

That last contrast is the headline capability, and the release is what makes it checkable. Training on five-second clips and interacting for hours is a generalisation claim, and a generalisation claim is exactly the kind that needs to be reproducible by someone other than its authors. Instantiating four models on three different backbones under shared interfaces is the other half of that argument: it separates what the recipe contributes from what any one architecture happens to be good at.

SolarWM. Open Data and Scalable Training for Long-Horizon Video World Models. By Junchao Huang and seventeen others, from the Chinese University of Hong Kong, Shenzhen, SLAI, and seven other institutions. Published on the second of September. This is the long version: the whole paper rather than the abstract.

Interactive video world models imagine future visual observations conditioned on camera motion, player actions or semantic instructions, which turns a passive video generator into an environment you can explore and control. The step this paper is about is the one from short-clip generation to long-horizon interactive rollout, and it identifies two requirements that turn out to be entangled with each other rather than separate engineering problems.

The first is coherent supervision across heterogeneous data sources. Existing datasets differ in temporal scale, visual quality, motion distribution, captioning style and camera conventions. Combine them naively and you get inconsistent supervision, so a model that performs well on individual sources deteriorates under multi-source training. The second is adapting diverse video generation backbones without compromising what they already learned. Video generators differ substantially in latent representations, attention structures and conditioning mechanisms. A shared adaptation strategy is desirable for scalability, and ignoring that heterogeneity compromises pretrained capabilities. But going the other way, with backbone-specific adaptation, limits systematic comparison and extensibility.

The data half is a reconfigurable multi-source engine, and what it produces is more interesting than how many clips it holds. One point four three million canonical clips from ten datasets are converted into a unified, frame-aligned contract covering visual observations, metric camera geometry, captions, quality metadata, selection decisions, and provenance. Two of those are unusual and are what make the mixture auditable rather than merely reproducible: recording which selection decisions were made, and where each clip came from. The engine also decouples source processing from mixture construction, so the two can vary independently, which is the concrete answer to the inconsistent-supervision problem: you can change what goes into a mixture without reprocessing the sources.

On the model side, four models spanning five to thirty-three billion parameters are instantiated on three different backbones, under shared camera-conditioning, training and inference interfaces, while preserving each one's native representations and objectives. The training recipe is three stages: bidirectional adaptation, then teacher-forced autoregressive initialization, then distribution matching distillation. That ordering is doing real work. The model starts as something that can attend in both directions, is then taught to run forward one step at a time with the ground truth available, and is finally distilled into something that can generate on its own at speed.

What the release actually contains is the evidence here, more than any single number. Data, pipeline, recipes, weights and framework are all published, which for a claim resting on generalisation across three backbones and a very long horizon is the difference between a result and a foundation someone else can build on. The four models span five to thirty-three billion parameters, which is enough range to show the recipe is not tuned to one capacity, and they come from three separate backbone families rather than three sizes of one.

There is no limitations section anywhere here, which for a release of this scope is a real gap. What can be established from the text is that everything is camera-conditioned and action-conditioned video, that the backbones are three specific families, and that long-horizon quality is asserted rather than quantified in the abstract. The word real-time is doing work that no reported latency figure supports there either. And the auditability the data contract provides is a property of the pipeline rather than evidence about the mixture: it records the selection decisions, and does not by itself show they were good ones.

The contribution to keep is the shape rather than any model. Coupling was the diagnosis, decoupling was the design, and the two components map onto the two halves of it: an engine that separates source processing from mixture construction, and a framework that separates shared interfaces from native internals. Both are the same move, applied once to data and once to models.