Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
Yugu Li, Zehong Cao, Peizhen Li, Yang Zhang, Siyi Hu, Jianglin Qiao
Adelaide University · University of North Texas · Curtin University · The University of Sydney
Published September 28, 2026 · on the feed September 29, 2026 · ▲ 6 · arXiv 2609.34848
The abstract
RLVR provides reliable trajectory-level credit, while OPSD offers dense supervision for token-level credit. This exposes a fundamental coupling when updating step-level credit direction and magnitude with teacher supervision, preventing steps from receiving reliable credit directions and contribution magnitudes, while making both vulnerable to teacher judgment errors and preference variance, as supported by our theoretical analysis. To separate credit direction from its contribution magnitude, we introduce Decoupled Credit Self-Distillation (DCSD), which theoretically decouples credit direction and magnitude into two reliable signals and uses them to calibrate privileged teacher supervision. Specifically, we design belief-margin probing to determine credit direction and marginal information gain to quantify credit magnitude, enabling step-to-token credit assignment for policy optimization. Across 11 benchmarks, DCSD achieves the best overall scores against GRPO, OPSD, RLSD, and RLCSD. Compared with base models, DCSD improves the overall score by 8.45 points on mathematical reasoning and 7.01 points on multimodal reasoning, while correcting the credit direction for 6\% of tokens and yielding a 1.5times reduction in token credit magnitude.
The Brief
2:10A short podcast summary of the paper. Written and voiced by AI.
Transcript
Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation. By Yugu Li and five others, from Adelaide University, the University of North Texas, and two other institutions. Published on the twenty-eighth of September. No code repository is linked to it.
Two training signals meet here, each strong where the other is weak. RLVR provides reliable credit for a whole trajectory, while OPSD offers dense supervision for credit at the level of tokens. Putting them together exposes a coupling. When a teacher's supervision updates both the direction of a step's credit and its magnitude at once, steps cannot receive a reliable direction or a reliable contribution size, and both become vulnerable to the teacher's judgment errors and to variance in its preferences. The authors support this with theoretical analysis. Hence the question in the title: can the teacher be trusted?
Decoupled Credit Self-Distillation, or DCSD, separates the direction of credit from the size of its contribution. It turns them into two reliable signals, which the authors argue theoretically are decoupled, and uses those signals to calibrate the privileged teacher's supervision.
Direction comes from belief-margin probing. Magnitude comes from marginal information gain, which quantifies how much a step contributes. Together they let credit be assigned from steps down to tokens for policy optimization. The order of operations is the point. A step first learns whether it helped or hurt, and separately how much it mattered, and only then is that credit passed down to the tokens inside it. A teacher that misjudges one of those two quantities no longer drags the other along with it.
Across eleven benchmarks, DCSD gets the best overall scores against GRPO, OPSD, RLSD and RLCSD. Against the base models, it raises the overall score by eight point four five points on mathematical reasoning and by seven point zero one points on multimodal reasoning. It also corrects the credit direction for six percent of tokens, and cuts token credit magnitude by one and a half times.
The teacher stays in the loop but loses its casting vote. Its dense signal is still used, while which way a step's credit points, and how much that step should count, are each settled by a measurement of their own.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is a teacher you can't fully trust.
You train a reasoning model with rewards from checking its final answer. That credit is reliable but blunt: a successful solution reinforces even its wrong or redundant steps.
The road ahead
- 1 · The problem ~7 min Rewards from checking only the final answer are reliable but blunt, and a token-scoring teacher adds detail but can misjudge every step.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Decide each step's credit direction and its size from two separate signals, then let the teacher only spread that credit over tokens.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Probe belief to set direction, measure new information to set magnitude, calibrate the teacher with both, then pass step credit down to tokens.
- 6 · The experiment ~6 min You’ll guess: Compared with the base models, by how many points did DCSD raise the overall score on mathematical reasoning?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.