Weaken to Strengthen: Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models

Weaken to Strengthen. Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models. By Yecheng Wu and two others, from the Massachusetts Institute of Technology and NVIDIA. Published on the twenty-eighth of October.

Self-distillation is a way to improve a reasoning model when no suitable external teacher is available. The common approaches, though, build a stronger self-teacher by relying on privileged information. That limits where they can be used. When all you have is unlabeled prompts, there is nothing privileged to build the stronger teacher from.

Weakening-Induced On-Policy Self-Distillation, or WISD, turns the direction around. Instead of constructing a stronger version of the model, it constructs a weaker counterpart of the original, and uses the contrast between the two as the training signal. No privileged information is required for that.

The paper explores two ways of making the weaker model. One is low-bit quantization. The other is magnitude pruning. Either way, the weakened copy comes from the original model itself, which is what lets the method run on unlabeled prompts alone. WISD is evaluated across diverse architectures and scales, on mathematical and on coding reasoning tasks.

Averaged across these benchmarks, quantization improves accuracy over the original models by four percentage points, and pruning by five point five points, which the authors report as state-of-the-art even without privileged information.

Two examples make it concrete. With pruning, Qwen3.5-4B rises from fifty-seven point seven percent to seventy point five percent on HMMT 2025. On LiveCodeBench v6 the same model rises from thirty-nine point one percent to fifty-six point seven percent. With quantization, Qwen3.6-35B-A3B goes from sixty-eight point five percent to seventy-four point six percent on HMMT 2025, and from sixty-one point eight percent to sixty-nine point five percent on LiveCodeBench v6.

The counterintuitive move is the one in the title. Most self-distillation reaches upward for a better teacher, and pays for it with information that is not always available. WISD reaches downward, degrading a copy of the model on purpose, and treats the gap between the original and the degraded copy as something to learn from.

Weaken to Strengthen. Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models. By Yecheng Wu and two others, from the Massachusetts Institute of Technology and NVIDIA. Published on the twenty-eighth of October. This is the long version: the whole paper rather than the abstract.

Self-distillation improves a model when there is no stronger teacher to learn from. The model generates its own solutions, and a version of itself grades them token by token. The catch is how that grading version is made stronger. Existing methods hand it something the student does not see: a reference solution, an expert demonstration, or feedback from the environment. That privileged information has to come from somewhere. Worked solutions can be costly to obtain, feedback needs an informative environment or verified rollouts, and when all you have is a pile of unlabeled prompts, none of it exists. Their question is whether a model can improve its reasoning by self-distillation with no privileged information at all.

Two ideas come first. The first is on-policy distillation. The student generates its own trajectories, and at every token a teacher says what it would have predicted there. Training pulls the student's next-token distribution toward the teacher's, on the states the student actually visits. The supervision is dense: every token gets a signal, not just the final answer.

The second idea comes from recent weak-to-strong work. Instead of one teacher, take a pair of frozen models and, at each token, subtract the log-probability under one from the log-probability under the other. That difference is a policy shift: it says which tokens to favour more and which less. Such a pair can supervise a student even when neither member is stronger than the student. Seen this way, the privileged-information methods are a pair too. The plain model is the weaker member, and the same model with a solution in its context is the stronger one.

The method is called WISD, Weakening-Induced On-Policy Self-Distillation. It keeps the original model as the stronger member and makes the weaker one by damaging a copy of its weights. Both see the same prompt and the same generated prefix. A token gets a positive contrast when the original model likes it more than the damaged copy does, and a negative contrast when the damaged copy likes it more. The contrast is not used raw. It tilts the original model's own distribution: each token's probability is scaled up or down by the exponential of its contrast, divided by a coefficient set to two, then renormalized. The student starts as the original model and is trained toward that tilted target. The pipeline is offline. The original model writes four responses to each of three thousand two hundred unlabeled prompts per domain, the frozen pair scores them once, and the student trains on that cache and is used alone at inference.

Two weakening operators are used, both training-free and needing no calibration data. They touch the attention and feed-forward projections, including mixture-of-experts expert projections, but leave embeddings, output heads, normalization and routing alone, so the two models' predictions stay directly comparable. Quantization rounds every weight to the nearest point on a symmetric grid whose step is set by the largest magnitude in its group. The default is three bits, which gives seven levels including zero. Pruning zeroes every weight at or below one sixth of the group's largest magnitude, then rescales each row back to its original length. That rescaling is a small but measured part of the recipe: in the appendix it adds about half a point on both models tested.

Five reasoning models are tested, from Qwen3-4B up to the mixture-of-experts Qwen3.6-35B-A3B, on three maths benchmarks and two versions of LiveCodeBench. Averaged over everything, quantization adds four points and pruning five and a half, with gains in most of the twenty-five model and benchmark settings. The largest jump is on Qwen3.5-4B, where pruning takes HMMT 2025 from fifty-seven point seven percent to seventy point five.

The comparison that tests the thesis most directly is in the analysis. Is the damaged copy doing real work, or is the student simply learning to trust its own confident tokens? Two baselines answer that on Qwen3-4B. One rewards tokens with the original model's own log-probability. The other contrasts the model with a softened copy of its own output distribution, with no weight damage at all. Both stay within a point of the original seventy-three point four on AIME 2024, while three-bit rounding reaches seventy-five point four. The dose matters too. Sweeping bit width on Qwen3.5-4B, eight bits gives about fifty-six percent on HMMT, three bits sixty-eight point three, and two bits collapses to forty-nine point nine. Pruning thresholds show the same hump.

The paper has no limitations section, so what follows is read off its own evidence. Each result is a single trained student with fixed seeds and no error bars, and several gains are under a point. Weakening does not always help: quantization left Qwen3.6-27B slightly below its original average. AIME 2024 doubled as the development benchmark for exploring constructions, and the sweeps that chose the defaults were scored on HMMT, which is also in the main table. The rival methods' scores are published numbers, gathered with different sample counts, temperatures and output limits from WISD's own evaluation, so beating them is a comparison across protocols.

The idea to keep is the reversal. Earlier self-distillation asks how to make a model smarter than itself, and needs outside information to do it. WISD asks how to make it worse, which is free. The analysis hints at where the useful contrast lands. Measured by gradient share, both constructions put about twenty-two percent of the signal on exploring solution paths and eleven percent on verification, more than those roles' share of the tokens. In the worked examples, the reinforced tokens are a digit in a paint-coverage calculation, the word not in the conclusion that ten gallons would not be enough, and a zero that rules out an invalid root. A broken copy of the model turns out to be a teacher that points at the checks.