G^2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
Ruikang Liu, Haoli Bai, Yuxuan Sun, Qian Zhang, Wenzheng Cai, Yanqi Hao, Feiyu Wang, Weidong Zhong and 3 others
ZTE Corporation · The Chinese University of Hong Kong · Northwestern Polytechnical University · Peking University · Nanjing University of Aeronautics and Astronautics
Published September 25, 2026 · on the feed September 29, 2026 · ▲ 4 · arXiv 2609.31009
The abstract
Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations. Methods with local, layer-wise objectives lack global supervision; while methods with global objectives fix their Hessian estimates at the start and ignore first-order gradients, so their guidance grows stale as quantization proceeds. This paper presents G^2PTQ, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective. By refreshing gradient and Hessian estimates before quantizing each Transformer block, G^2PTQ avoids the staleness of prior global methods. Furthermore, to stabilize the exact first-order compensation, we introduce a trust-region scaling mechanism that dynamically bounds the gradient step to prevent exploding weight updates. Finally, we derive efficient implementations for block-wise Hessian approximation and exact gradient compensation. Experimental results on various model families and bit-widths demonstrate that G^2PTQ enables better alignment with the full-precision model, outperforming state-of-the-art baselines. Code is available at: https://github.com/G2PTQ/G2PTQ.
The Brief
2:15A short podcast summary of the paper. Written and voiced by AI.
Transcript
G^2PTQ. Improving LLM Post-Training Quantization with Generalized Gradient Compensation. By Ruikang Liu and ten others, from Z T E Corporation, the Chinese University of Hong Kong, and three other institutions. Published on the twenty-fifth of September. Its code repository is worked out: about seventy-five source files, with a readme and a licence.
Post-training quantization shrinks the memory and compute footprint of a large language model without retraining it, and methods built on GPTQ have become the de facto standard. The paper finds two limitations in them, and they pull in opposite directions. Methods with local, layer-wise objectives have no global supervision. Methods with global objectives do have it, but they fix their Hessian estimates at the start and ignore first-order gradients altogether. As quantization proceeds and the model changes, that guidance goes stale.
G^2PTQ is a unified framework built around what the authors call Generalized Gradient Compensation. It brings first-order and second-order information together, under an optimization objective that is supervised globally but carried out block by block. The fix for staleness is direct: before each Transformer block is quantized, the gradient and Hessian estimates are refreshed, so every block is guided by information about the model as it currently stands.
Using the exact first-order compensation brings its own risk, which is that weight updates can explode. To keep it stable, G^2PTQ adds a trust-region scaling mechanism that dynamically bounds how large the gradient step can be. The authors also derive efficient implementations for the two expensive pieces, the block-wise Hessian approximation and the exact gradient compensation, so that refreshing at every block remains practical.
Experiments cover various model families and bit-widths. Across them, the abstract reports that G^2PTQ aligns more closely with the full-precision model and outperforms state-of-the-art baselines. It gives no specific figures, so the size of that margin is not something this summary can state.
The two limitations the paper diagnoses mirror each other. Local methods lacked a global view, and the global methods had one that was out of date. G^2PTQ answers each in turn: it keeps the global view and renews it at every block, with first-order information included and a bound in place to stop that information from overshooting.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Shrinking a model without retraining is a balancing act, and the guidance keeps going stale.
You are compressing a large language model to a few bits per weight without retraining. The standard recipe quantizes weights and adjusts the rest to compensate, one layer at a time.
The road ahead
- 1 · The problem ~6 min Shrinking a language model's weights without retraining needs error compensation, but local fixes miss the whole model and global guidance goes out of date.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Refresh gradient and Hessian estimates before each block, compensate with both, and cap the gradient step so updates stay stable.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min For each block: refresh gradient and Hessian, compensate quantization error with both, and scale the gradient step to stay in a trust region.
- 6 · The experiment ~6 min You’ll guess: Across model families and bit-widths, how did G^2PTQ's quantized models compare with state-of-the-art post-training quantization baselines?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.