PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference
Gil Jae Song, Sungjoo Yoo
Seoul National University
Published October 14, 2026 · on the feed October 16, 2026 · ▲ 4 · arXiv 2610.19605
The abstract
Long-output inference in large language models suffers from severe KV cache memory and bandwidth bottlenecks. While KV cache quantization alleviates this issue, low-bit quantization (e.g., 2-bit) severely degrades model accuracy. Recent mixed-precision key-cache quantization methods mitigate this degradation by allocating higher precision to important key channels, but estimate channel saliency using only key statistics. Through attention logit perturbation analysis at the RoPE channel-pair level, we show that quantization errors in channel pairs with large query--key pair-norm products contribute significantly to logit variation. Based on this insight, we propose PN-Mix, a mixed-precision key-cache quantization method for RoPE-based decoder-only LLMs, which allocates higher precision to salient channel pairs identified once during the prefill stage and reused throughout decoding. Across standard reasoning, code-generation, and AIME benchmarks, PN-Mix with 2.26-bit average precision achieves comparable or better average accuracy than prior mixed-precision methods using 2.32--2.39 bits. Furthermore, in our Pareto sweep, PN-Mix degrades gracefully down to 1.5 bits, whereas the compared baselines exhibit sharper performance drops below 2.25 bits.
The Brief
1:58A short podcast summary of the paper. Written and voiced by AI.
Transcript
PN-Mix. Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference. By Gil Jae Song and one other, from Seoul National University. Published on the fourteenth of October. No code repository is linked to it.
When a large language model produces long outputs, its key-value cache becomes a severe bottleneck for memory and for bandwidth. Quantizing the cache helps, but going to very low bit widths, two bits for example, badly hurts accuracy. Recent mixed-precision methods for the key cache soften that damage by giving higher precision to the key channels that matter most. Their weakness is how they decide what matters: channel saliency is estimated from key statistics alone.
The authors look at the question from the attention logits instead. Working at the level of RoPE channel pairs, they perturb the logits and find that quantization errors in pairs where the product of query and key pair norms is large contribute significantly to how much the logits vary. Saliency, in other words, depends on the query as well as the key.
PN-Mix turns that into a mixed-precision key-cache quantization method for RoPE-based, decoder-only models. Salient channel pairs receive higher precision. They are identified once, during the prefill stage, and the same choice is reused for the whole of decoding.
Tests span standard reasoning, code generation and AIME benchmarks. With an average precision of two point two six bits, PN-Mix reaches average accuracy comparable to or better than prior mixed-precision methods that use between two point three two and two point three nine bits. A Pareto sweep pushes further: PN-Mix degrades gracefully down to one point five bits, while the compared baselines show sharper drops once they go below two point two five bits.
The argument runs from a measurement to a rule. Errors hurt most where query and key pair norms are jointly large, so that is where the bits should go. The choice is made once, at prefill, and decoding simply inherits it.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free