WorldAttention: An Efficient Attention Architecture for Interactive Video World Models

WorldAttention. An Efficient Attention Architecture for Interactive Video World Models. By Zeyu Zhang and eight others, from DAMO Academy, Alibaba Group, Zhejiang University, and three other institutions. Published on the twenty-eighth of September. Its code repository is worked out: about seventy-five source files, with a readme and a licence.

Interactive video world models built on autoregressive diffusion try to simulate environments that stay coherent over time while following text instructions. For embodied AI and for planning by simulation, they need low latency and long durations at once. Current frameworks mostly keep cost bounded with a sliding window, and that throws away history, which undermines long-range interaction. A model that can only see the last few moments cannot keep faith with something that happened long before, and in an interactive world that is exactly what a user expects it to do. Keeping the full history is the obvious alternative, and it is prohibitive. Attention's quadratic complexity drives up computation, and the KV cache grows linearly until GPU memory saturates. One road forgets, the other runs out of room, and long-duration, low-latency generation needs neither.

WorldAttention treats this as a systems problem. It is an attention architecture that gets its efficiency from co-designing two things: the attention kernels themselves, and the way the KV cache is managed across a hierarchy of memory. The paper presents them as a pair: one decides how attention is computed over the past, the other decides where that past is kept and how it is fetched.

The first piece is Hybrid Sparse Attention, HSA. It combines linear global attention with sparse attention that adapts per head, so the model keeps a cheap view of everything plus selective detail. The second piece is a Hierarchical KV Cache, HKV. Historical key and value pairs are organized into pages indexed by meaning, spread across several tiers of memory. That allows fine-grained retrieval of the past, and it puts the system in control of what stays resident on the GPU. Both designs come with tailored kernels, because theoretical efficiency only counts if it turns into real-world speed.

Extensive experiments on VBench-Long and InterVBench show WorldAttention consistently ahead of prior state-of-the-art methods. Subject consistency reaches zero point nine four seven two on VBench-Long. On InterVBench it reaches zero point nine six six eight. Subject consistency is the measure singled out, which fits the aim: a world model that remembers should keep its subjects the same over a long run.

The trade the field had accepted was memory or history, pick one. This architecture refuses that trade at the level of hardware: a cheap global view does the heavy lifting, and the long past moves off the GPU into indexed pages that can be brought back when needed.