Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu and 5 others
Nanjing University · JD.com
Published October 9, 2026 · on the feed October 15, 2026 · ▲ 243 · arXiv 2610.13521
The abstract
Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through dense token-level supervision on student-generated trajectories. However, domain routing only distinguishes the supervision sources for different domains; updates from all domains still act on shared parameters. When teachers differ substantially in response style and domain expertise, supervision that is effective in isolation may still lead to persistent gradient conflicts and capability degradation during joint optimization. To address this challenge, we propose Cross-Teacher Guarding for MOPD (Road MOPD), which introduces non-routed teachers as auxiliary cross-domain critics on the same student trajectories and coordinates multi-teacher supervision through token-level credit assignment and gradient updates. Specifically, cross-teacher-guided credit assignment uses teacher agreement to strengthen token-level corrections jointly supported by the teachers and attenuate conflicting signals; preventive multi-teacher gradient projection uses auxiliary gradients from the same trajectories to constrain potentially conflicting updates and adaptively controls projection strength to retain target-domain learning signals. In experiments, Road MOPD mitigates degradation in the disadvantaged domain, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
The Brief
2:09A short podcast summary of the paper. Written and voiced by AI.
Transcript
Beyond Domain-Routed Supervision. Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation. By Shi-Yu Tian and twelve others, from Nanjing University and JD.com. Published on the ninth of October. No code repository is linked to it.
In on-policy distillation, a student generates its own trajectories and a teacher supplies dense, token-level supervision on them. The multi-teacher version, MOPD, routes the data from each domain to a specialized teacher, and so integrates several teachers' capabilities into one student. The catch is that routing only separates the sources of supervision. The updates from every domain still land on the same shared parameters.
That matters when the teachers differ a lot, in response style and in domain expertise. Supervision that works well in isolation can then still produce persistent gradient conflicts when everything is optimized jointly. One domain's progress pulls against another's, and capabilities degrade.
Cross-Teacher Guarding for MOPD, called Road MOPD, brings in the teachers that were not routed to a given domain. They act as auxiliary critics from other domains, looking at the same student trajectories. Their views are then used to coordinate the supervision, both in how credit is assigned to tokens and in how gradients are updated.
There are two parts. Cross-teacher-guided credit assignment uses agreement among teachers. Token-level corrections that the teachers jointly support are strengthened, and conflicting signals are attenuated. Preventive multi-teacher gradient projection works on the updates themselves. It uses auxiliary gradients from those same trajectories to constrain updates that might conflict, and it adapts how strongly it projects, so that the learning signal for the target domain is retained rather than projected away.
In experiments, Road MOPD mitigates the degradation in the disadvantaged domain, the one that was losing out. Under high conflict, it gains more than five percentage points. It also improves overall performance in settings with low conflict, and in settings with three domains.
Routing decides who teaches what, but not where the lessons collide. By letting the other teachers weigh in on every trajectory, this approach handles the collision at the level of tokens and gradients, inside the shared parameters where it actually happens.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Many teachers, one student, one set of weights. Something has to give.
You are training one student model with several teacher models, a math teacher, a code teacher and a medical teacher, each giving token-by-token feedback on the student's own answers in its domain.
The road ahead
- 1 · The problem ~6 min Giving each domain its own teacher does not stop the domains' updates from fighting, because they all change the same student weights.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Let the other domains' teachers critique the same student answers, use their agreement to weight tokens, and trim conflicting parts of each update.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Route data to its teacher, add other teachers as critics on the same answers, weight tokens by agreement, then project away conflicting update parts.
- 6 · The experiment ~6 min You’ll guess: Under high conflict between domains, how many percentage points did Road MOPD gain?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.