SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China
Published November 10, 2026 · on the feed November 12, 2026 · ▲ 33 · arXiv 2611.13410
The abstract
Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce SteerDistill, a quantization-aware distillation method in which a full-precision teacher guides the student to revise failed responses and iteratively refines its guidance using feedback from student execution. This closed-loop teacher steering produces locally verified behavioral corrections. We internalize these corrections by training the student without guidance to match the steered student's predictions at corrected turns and the full-precision teacher's at other turns, enabling unassisted inference. Across five agentic benchmarks and three models, SteerDistill exceeds the strongest quantized baseline by 1.67--2.18 points in average score under W4A16 and stays within one point of BF16. On τ^2-Bench, it outperforms QeRL by 1.84 percentage points in success rate using 67% fewer training GPU hours and produces shorter successful trajectories than the quantized baselines.
The Brief
2:14A short podcast summary of the paper. Written and voiced by AI.
Transcript
SteerDistill. Quantizing LLM Agents via Closed-Loop Teacher Steering. By Genming Cui and five others, from the Institute for AI Industry Research at Tsinghua University and the University of Science and Technology of China. Published on the tenth of November.
Low-bit quantization makes language-model agents cheaper to run, but it can cost them task success. Looking at the rollouts where quantized agents fail, the authors find a pattern. The agent overlooks a task requirement or a condition in its environment, and the environment transitions it produces drift away from those of a successful full-precision reference. Correcting that drift is harder than it sounds. Offline data may never visit the contexts where this particular student fails. And the student's own rollouts, in exactly those contexts, may contain few correct responses to learn from.
SteerDistill is a quantization-aware distillation method built around a closed loop. A full-precision teacher guides the quantized student to revise its failed responses. The student then executes, and the teacher uses the feedback from that execution to refine its guidance, round after round. What comes out are behavioral corrections that have been verified locally, in the very contexts where the student went wrong.
Those corrections still lean on the teacher being there, so the last step is to internalize them. The student is trained without any guidance to match two targets. At the turns that were corrected, it matches the predictions of the steered student. At every other turn, it matches the full-precision teacher. The result is a student that needs no assistance at inference time.
The evaluation spans five agentic benchmarks and three models. Under W4A16 quantization, SteerDistill exceeds the strongest quantized baseline by between one point six seven and two point one eight points in average score, and it stays within one point of BF16. On tau-squared Bench, it outperforms QeRL by one point eight four percentage points in success rate, while using sixty-seven percent fewer training GPU hours. Its successful trajectories are also shorter than those of the quantized baselines.
The method is built as an answer to its own diagnosis. Failures came from contexts the student could not learn its way out of alone, using either offline data or its own attempts. So the teacher is brought into precisely those contexts, steers until a correction is verified, and then is distilled away. Cheaper weights, with the full-precision behavior pulled back at the turns where it had slipped.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Cheaper agents, worse agents. That is the wall here.
You have squeezed a tool-using language agent into low-bit weights to cut its running cost. Now it fails tasks the full-precision version solves.
The road ahead
- 1 · The problem ~6 min A low-bit agent fails at specific turns where it overlooks a requirement, and neither offline data nor its own retries supply the right fix there.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min A full-precision teacher steers the student to fix its own failed turns, keeps locally checked fixes, then trains them into the student so it works alone.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Each training cycle collects failed rollouts, steers and verifies revisions in a closed loop, then distills the kept corrections into the unguided student.
- 6 · The experiment ~6 min You’ll guess: On tau-squared Bench, how did SteerDistill's training GPU hours compare with the reinforcement-learning baseline QeRL's?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.