Speculative Forcing for Efficient Autoregressive Video Generation

Speculative Forcing for Efficient Autoregressive Video Generation. By Hyunwook Choi and three others, from KAIST and NAVER AI Lab. Published on the fifteenth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Few-step autoregressive video generators make streaming generation possible, but they send every token through the final denoising step, however much refinement that token actually needs. The authors find that this uniform computation is often unnecessary. Many tokens already closely match their eventual full-step output within the initial steps. Others genuinely need refinement until the very end. Spending the same effort on both is where the waste lies.

Speculative Forcing treats the intermediate clean estimates as drafts, and predicts when an individual token can safely be accepted early. Once accepted, a token is frozen as its final output and receives no further denoising. Its cached keys and values, though, stay available as context for the tokens still being refined, so freezing one token does not cut the others off from it.

Deciding acceptance falls to a verifier, and two alternative designs are explored. One is training-free: it judges a token by its similarity to clean content and by its stability across steps. The other is learned: it reads the frozen generator's intermediate features and estimates how far each token still is from its full-step output. The first asks whether a draft already looks finished and has stopped changing. The second asks the generator itself, by way of its internal features, how much work remains. Either way, the generator stays frozen; only the decision of when to stop is added.

The method is tested across short-video, long-video and world generation, and in each it preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator using about one point seven denoising steps per token, and runs more than one point five times faster, at forty-one point seven frames per second in one setting and thirty-seven point eight in another.

Lingbot-World two point zero is the harder case, because camera motion keeps changing what is visible. There, with more of each frame in flux, it maintains quality with about two point five denoising steps per token, and runs one point three five times faster, at twenty-four point five frames per second.

The saving comes from noticing that the denoising budget was spread evenly over tokens that needed it unevenly. Accepting a token early is a bet, and the verifier is what decides when the bet is safe. Because a frozen token keeps serving as context, the tokens still in progress lose nothing by its early exit.