PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

PrismQuant. Optimal Null-Space Rotations for Grouped Quantizers. By Yanlong Chen and four others, from Nanyang Technological University, Nanjing University of Posts and Telecommunications, and two other institutions. Published on the twenty-sixth of September. Its code repository is public and real: about twenty-five source files.

A common working assumption in low-bit quantization is that smaller activation outliers lead to better results. PrismQuant opens by saying that this is not enough. Smaller outliers do not necessarily mean better quantization, because what also matters is how the activations are aligned with the quantizer itself.

PrismQuant is a quantizer-aware rotation framework. It aligns the leading eigenspace of the activations with the constant group subspace of asymmetric grouped INT4. That subspace is a good destination for a specific reason: the affine offsets can represent the energy that lands in it, without widening the range inside the group. The authors frame the design of the rotation as a Ky Fan trace maximization, and derive a closed-form solution that is provably optimal for this alignment objective.

In practice the rotation is built from compact Householder transformations, using their compact WY representation. That makes construction gradient-free and application efficient, both at foldable sites and at sites where the rotation is applied online. A predictive range law completes the picture. It connects the activation energy that remains unaligned, together with the group size, to the variation that matters for quantization.

Experiments cover Llama, Qwen, and Mistral, with dense models up to seventy billion parameters as well as a thirty billion parameter mixture-of-experts model. With weights, activations, and the KV cache all at four bits, PrismQuant sets the state of the art among the compared methods on Llama three point two at three billion, in both perplexity and accuracy.

On Llama three point one at seventy billion, it reaches a perplexity of three point eight five and an average zero-shot accuracy of seventy-two point four six percent. That is only zero point two two percentage points below full precision.

The deployment study uses Llama three point one at eight billion. There the optimized implementation speeds up prefill by one point five one times, and CUDA Graph decode by one point two two times, over matched FP16 baselines. Decode peak memory drops by fifty-six point three four percent. The price is two point three five percent of additional Graph decode latency compared with Hadamard.

The thesis fits in a sentence: stop asking only how small the outliers are, and ask where the energy sits relative to the quantizer's groups. Once the question is posed that way, finding the best rotation stops being a search and becomes a formula.