QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching

QuantMLA. Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching. By Zunhai Su and seven others, from the University of Hong Kong, the Meituan LongCat Team, and two other institutions. Published on the thirtieth of September. No code repository is linked to it.

Multi-Head Latent Attention, or MLA, already keeps its cache compact. It stores two things: a cache for the content path, and a separate cache for the decoupled RoPE path that carries position. Even so, cache memory still grows linearly with context length and with batch size. Quantizing that cache to low bit widths is the obvious lever, but the two paths do not respond to error in the same way. Errors on the RoPE path in particular are amplified sharply in the attention output.

The paper starts by building a systematic model of these dual-path quantization errors. It characterises how each path distorts the attention output differently, and it explains why the RoPE-path errors are amplified so much more than the content-path ones.

Guided by that analysis, QuantMLA treats the two paths as two problems. It is a function-aligned framework for low-bit dual-path quantization: each path gets its own transformation, and each transformation is learned against an objective that matches what that path actually does inside attention, rather than against a generic reconstruction of the stored numbers.

First, the authors derive path-specific transformation spaces. Transformations in these spaces preserve the full-precision computation, and they can be fused into the model parameters offline, so there is no transformation overhead at inference time. Within those spaces, the content path is trained on attention-output reconstruction, which captures its coupled matching and aggregation errors. The RoPE path is trained on positional QK reconstruction, which preserves the RoPE-induced component of the attention logits and admits a theoretical bound on output distortion. On the systems side, a native low-bit MLA attention kernel folds unpacking and dequantization directly into the attention computation.

Across four MLA model families, QuantMLA enables what the authors believe is the first reported joint INT4 caching of both the content cache and the RoPE cache, with minimal accuracy degradation. Pushing further, the content cache can go down to INT2 while the RoPE key cache stays at INT4, and performance remains competitive on challenging reasoning and code benchmarks.

The physical cache layout gives three point five nine times compression at a context of one hundred and twenty-eight thousand tokens. Under a serving workload built around cache pressure, whole-job output throughput is five point one six eight times higher than with BF16. Code is promised upon acceptance.

The argument runs from diagnosis to design. Once the error model shows that the position path and the content path fail differently, quantizing them with one shared recipe stops making sense, and giving each path an objective shaped like its own role is what lets both caches drop to four bits together.