Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation

Knowing When to Teach. Learning from Teachable Prefixes in Multilingual Reasoning Distillation. By Jaewon Cho and six others, from KAIST and Korea University. Published on the fifth of November.

On-Policy Self-Distillation, or OPSD, trains a student on its own reasoning. The teacher is the same model, conditioned on a reference solution. Carrying this over to multilingual reasoning asks a lot of that teacher: it has to give useful feedback in every language, including when the student has already made mistakes partway through. The authors observe a failure they call Teacher Drift. As the student's errors pile up, the teacher may start following those incorrect steps instead of correcting them. Their analysis finds that this shift tends to happen earlier in lower-resource languages. Put differently, the reference solution is supposed to anchor the teacher, but once the trajectory has wandered far enough, the teacher anchors to the student instead. And how soon that happens is not the same from one language to another.

Teachable On-Policy Self-Distillation, or TOPSD, responds by deciding, for each reasoning trajectory, where teacher supervision should apply and whether it should apply at all. Rather than trusting the teacher over the whole trajectory, it supervises only a portion the teacher can still handle.

Two steps do this. First, TOPSD uses the likelihood the teacher and the student assign to the trajectory to estimate where drift begins, and selects a prefix up to that point for supervision. Second, it checks whether the teacher can recover the correct answer starting from that prefix. Distillation is applied only to prefixes that pass this check, so a prefix the teacher cannot rescue is not used to teach.

The experiments cover multilingual mathematical reasoning across ten languages and three model scales. TOPSD consistently outperforms conventional OPSD, and the advantage is most visible in low-resource and medium-resource languages. The comparison is with the same self-distillation recipe applied without the prefix selection and the verification step.

That pattern fits the diagnosis. Drift arrived earliest in lower-resource languages, and that is where restricting supervision to verified prefixes helps most. The underlying idea is modest and portable: a teacher is only as good as the point it is asked to teach from, so find where that point stops being trustworthy and teach only before it.