Few-Step Streaming Video Generation via Condition Shifting

Few-Step Streaming Video Generation via Condition Shifting. By Deyuan Liu and ten others, from Tsinghua University, Shengshu Technology, and one other institution. Published on the tenth of November.

Generating a video in a few sampling steps makes streaming generation cheaper. Training a generator to work in so few steps is where the cost reappears. Distribution Matching Distillation, or DMD, supervises the few-step generator with a separate fake-score network. That second network adds training memory and computation of its own, and because the generator keeps changing during training, the fake network has to keep tracking it.

The authors extend APEX to streaming video, and the move that makes it work is condition shifting. Instead of a second network, the velocity field of the generated distribution is estimated within the generator itself. One set of parameters plays both roles, and the condition decides which. The original text condition selects generation. A shifted condition selects fake estimation.

With that in place, the independent fake network is gone. Kept from S2-Avatar are its frozen reference teacher, Self-Replay Forcing, and perceptual regularization. The evaluation runs on videos one hundred and eighty seconds long, with three backbone evaluations per block and a one-step spatial Refiner. Two comparisons sit on top of that: a human study of seventy cases, and a StreamAV-Bench comparison of three hundred and twenty cases.

Against S2-Avatar on the benchmark, APEX scores higher on video instruction fulfillment on the Progressive track: three point eight five six against three point eight three one. Visual aesthetics is higher on both tracks, at roughly zero point six four two against zero point six three three.

The human raters separate the systems more clearly. Their strongest preference for APEX is on instruction following, a net thirty percent. Consistency and video quality follow, each at four point three percent. Not every axis moves the same way. Motion quality favors S2-Avatar, audiovisual synchronization is predominantly tied, and usability is unchanged. On cost, separate measurements under matched training settings put peak allocated GPU memory per GPU four point four percent lower than DMD, and mean iteration time nine point two percent shorter.

Where DMD needed two networks, APEX asks one network two different questions, and lets the condition say which question is being asked. The return on that is concentrated: a clear lead in instruction following, small edges in aesthetics, consistency and quality, a loss on motion, and a training run that uses somewhat less memory and time.