HLA-WM: Hybrid Linear Attention for Long-Horizon Video World Models

HLA-WM. Hybrid Linear Attention for Long-Horizon Video World Models. By Zhuokun Chen and six others, from Monash University, the University of Adelaide, and one other institution. Published on the fifth of October. Its code repository is worked out: about two hundred source files, with a readme.

A video world model generating over a long horizon needs persistent memory. Without it, a scene revisited late in a rollout will not stay consistent with the same scene seen earlier. Two kinds of attention offer opposite trades. Softmax attention keeps the full generation history in a KV cache that keeps growing. Recurrent linear attention compresses history into fixed-size states, at substantially lower memory cost. The authors identify what that compression costs in one such model, Gated DeltaNet, or GDN: severe long-range forgetting. Information from a distant but relevant scene is progressively attenuated by every state update that follows it.

HLA-WM is a hybrid that keeps recurrent linear attention but lets its memory be addressed selectively. It combines coarse-grained retrieval, guided by geometry, with fine-grained computation over recurrent linear states. It is also training-free. The framework is applied to an existing autoregressive generator, and nothing is retrained.

It works by exploiting the affine structure of GDN. That structure allows compact, chunk-wise transition summaries to be cached. During generation, camera geometry is used to retrieve the historical chunks relevant to the current scene, and their summaries are recomposed into a recurrent state specific to the query. Instead of one state into which all of history has been folded, each query gets a state assembled from the parts of history that concern it.

On the sixty-second SANA-WM-Bench, HLA-WM improves all six aggregate metrics of revisit consistency and camera control over the base generator. They include a gain of zero point seven four decibels in PSNR, and a twenty-eight point five percent reduction in rotation error. The improvements persist after downstream refinement. They also generalize to MBench-A, where HLA-WM consistently improves all three revisit-consistency metrics, across all four subsets and every evaluated inference mode, over five hundred and forty-seven samples.

Then the cost. At a sixty-second context, historical-state memory is twelve times smaller than with full KV caching, and inference throughput drops by at most one point six percent.

The authors read this as showing that selectively addressable recurrent memory can improve long-range scene recall while keeping the efficiency advantages of GDN. The choice between remembering everything expensively and forgetting cheaply turns out not to be forced. A fixed-size memory can still be searched, provided what it stores is kept in pieces that can be found again and put back together on demand.