Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Xingyu Zhu, Pu, Yi, Ziheng Cheng, Ang Lv, Jing Liu, Lexing Ying, Yiyuan Ma and 1 others
ByteDance Seed · Princeton University · Stanford University · University of California, Berkeley
Published September 28, 2026 · on the feed September 30, 2026 · ▲ 96 · arXiv 2609.36322
The abstract
Chunked KV-cache compression reduces the memory and attention costs of long-context inference by compressing windows of consecutive tokens into fewer cache entries at a fixed stride. Such compression also introduces a new positional coordinate: a token's phase, or its position relative to compression-window boundaries. We uncover a systematic asymmetry in models using such compression: the same information can be easy to retrieve at one phase and difficult at another. We call this periodic variation in retrieval performance phase sensitivity. In large open-weight models with such compression, long-context retrieval accuracy can differ by up to 40 percentage points across phases, revealing periodic weak spots that average benchmark scores can conceal. To investigate this behavior, we pretrain a family of transformers from scratch across multiple KV-compression designs, reproducing phase sensitivity across the variants. Mechanistic analysis using causal interventions in these models reveals phase specialization: different attention components contribute asymmetrically to retrieving information at different source phases. We further analyze idealized retrieval models, showing how gradient flow dynamics may favor sharp phase specialization. Evaluating models with chunked KV-cache compression thus requires measuring across compression phases: high average accuracy can coexist with systematic positional failures.
The Brief
2:27A short podcast summary of the paper. Written and voiced by AI.
Transcript
Periodic Weak Spots. Phase Sensitivity from Chunked KV-Cache Compression. By Xingyu Zhu and eight others, from ByteDance Seed, Princeton University, and two other institutions. Published on the twenty-eighth of September. No code repository is linked to it.
Chunked KV-cache compression makes long-context inference cheaper. Windows of consecutive tokens are compressed into fewer cache entries at a fixed stride, which cuts both memory and attention costs. The claim here is that this compression also introduces a new positional coordinate: a token's phase, meaning its position relative to the boundaries of a compression window. And phase matters. The same information can be easy to retrieve at one phase and difficult at another. The authors call this periodic variation in retrieval performance phase sensitivity. Picture a fact placed early in a window, then the same fact placed late in the next one. Nothing about the fact has changed, only where it falls against the stride, and yet the model's chance of finding it again can be quite different. Because the windows repeat, so does the pattern of strong and weak positions.
In large open-weight models that use such compression, long-context retrieval accuracy can differ by up to forty percentage points across phases. Those are periodic weak spots, and an average benchmark score can conceal them.
To study the behaviour directly, the authors pretrain a family of transformers from scratch across multiple KV-compression designs, and phase sensitivity reappears across the variants. Causal interventions in those models then reveal phase specialization: different attention components contribute asymmetrically to retrieving information from different source phases. An analysis of idealized retrieval models offers a possible reason, showing how gradient flow dynamics may favour sharp phase specialization.
The finding belongs to models that use chunked compression at a fixed stride. Phase is a coordinate that only this kind of compression creates. The forty point gap is a ceiling, stated as up to, in large open-weight models, not a constant across every model and position. And the gradient flow account is given for idealized retrieval models, as something the dynamics may favour, rather than as a proof about the large models themselves.
For anyone evaluating these models, the consequence is procedural. Measure across compression phases, because high average accuracy can coexist with systematic positional failures. A single score averaged over every position is precisely where a periodic weak spot hides. Compression bought a cheaper cache, and it also quietly bought a new axis along which a model can fail.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model that looks fine on average. Look closer and something repeats.
You serve a long-context model that saves memory by squeezing every few consecutive tokens into fewer cache entries, window after window, at a fixed stride.
The road ahead
- 1 · The problem ~7 min A model that squeezes fixed windows of tokens into its memory can answer differently when the same prompt merely slides over by a token or two.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Fixed compression windows give every token a phase, its offset inside its window, and retrieval quality rises and falls with that phase.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Measure retrieval by phase in large models, rebuild the effect in controlled small models, find which heads cause it, then explain it with theory.
- 6 · The experiment ~6 min You’ll guess: In large open-weight models using chunked KV-cache compression, how much can long-context retrieval accuracy differ depending only on where the information falls relative to compression-window boundaries?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.