QuantWM: Temporally Consistent 2-Bit KV Cache Quantization for Video World Models

QuantWM. Temporally Consistent 2-Bit KV Cache Quantization for Video World Models. By Jiaqi Zhao and five others, from the Harbin Institute of Technology, Shenzhen and the National University of Singapore. Published on the twenty-eighth of September. No code repository is linked to it.

Video world models keep long-range temporal consistency by storing a KV cache as they generate. The longer generation runs, the larger that cache grows, and KV cache memory becomes a major bottleneck for deploying these models. Low-bit quantization is the natural response. Existing two-bit KV cache quantization methods can be nearly lossless on VBench. Applied to video world models, though, the authors find they still cause severe temporal flickering and visual degradation.

A closer look turns up something surprising. Key quantization produces smaller reconstruction errors than Value quantization, yet it leads to larger degradation in the output. The authors trace that discrepancy to attention. A perturbed Key can change the attention logits, and that shifts which temporal-spatial tokens a Query selects. A small error in the wrong place, in other words, can move where the model looks.

QuantWM follows from that diagnosis. It is a training-free, two-bit KV cache quantization framework for video world models, and its aim is to preserve the attention logits and the temporal-spatial token selection while the cache is being quantized. The focus moves from how large the errors are to what they do to attention. Two complementary techniques are introduced to mitigate the attention shifts.

The first is quantization-sensitivity-aware clustering, or QSAC. It selects INT2-friendly Key centroids by jointly considering two things: the historical sensitivity of Queries, and the residual ranges. The effect is to reduce quantization error in the channels that are more critical to attention. The second is principal-subspace attention compensation, or PSAC. Whatever Key error remains, PSAC restores along the dominant Query subspace using low-rank projections, which gives a direct and efficient correction that stabilizes the attention logits. One technique decides where error does the least harm. The other repairs what is left, along the directions the Queries mostly occupy.

Experiments cover five models: LingBot-World-v2, HY-World one point five, Matrix-Game-2, Longcat-Video and Causal-Forcing. Across them, QuantWM significantly improves visual quality and temporal consistency, and it outperforms existing methods across benchmarks. It reaches up to six point two times KV cache memory compression, with limited additional overhead.

The lesson of QuantWM sits in its diagnosis as much as in its compression ratio. A quantizer can look nearly lossless on a standard benchmark and still make a world model flicker, because the error that matters is not the one reconstruction measures. Key errors are the smaller ones and still do more damage, since the Key shapes where attention goes. Building the quantizer around that fact is what lets a two-bit cache keep a video world model temporally consistent.