DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation

DuoOPD. Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation. By Ao Yu and eight others, from the University of Science and Technology of China, the Hong Kong Polytechnic University, and two other institutions. Published on the twenty-seventh of September. Its code repository is worked out: about nine hundred source files, with a readme and a licence.

On-policy distillation, OPD, trains a student on its own responses, with token-level feedback from a stronger teacher. Two facts complicate that. The teacher can fail on questions the student already gets right. And how often each model succeeds varies from task to task. Plain OPD ignores these outcomes, and on average it pushes down even the student's correct responses. Gating the feedback on whether the student was correct fixes the direction of the push, but it still uses the teacher the same way whether the teacher succeeded or not. A teacher that got the question wrong is treated exactly like one that got it right, and in a multi-task mix, where the balance of successes shifts from task to task, that sameness is what this work sets out to change.

DuoOPD splits the job in two. The student's own outcome sets the direction of the feedback. The joint outcome, teacher and student together, decides how the teacher supports it. Direction answers whether a response should be pushed up or down. Support answers what the teacher contributes to that push, and the answer depends on whether the teacher itself got the question right.

Consider the two lopsided cases. When only the teacher succeeds, its verified answer becomes context for scoring the student's failed response. When only the student succeeds, a weight shared within the task reinforces the student's whole response. In the first case the teacher's success is put to work as a reference. In the second, where the teacher failed, what gets reinforced is the student's own success, taken as a whole response. One rule covers all four combinations of outcomes, with no settings tuned per task.

Across Qwen3 and Llama, DuoOPD beats all five baselines in mean macro accuracy. Over plain OPD the reported improvements are two point five eight percentage points, and five point nine eight percentage points. It also leads on two further task mixtures, which span scientific calculation, instruction following, and code generation.

The ablations separate the two ingredients. Setting direction by outcome, on its own, stays close to the gated baseline. Most of the gain comes from the designs that use the joint outcome.

Who got it right matters, and so does who got it right together. A teacher that failed should not be leaned on in the way a teacher that succeeded can be, and a student that succeeded alone has earned reinforcement rather than correction. The ablation puts the weight of the result on that second idea.