MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation
Tianze Xu, Yanzhao Zheng, Zhentao Zhang, Yuanqiang Yu, Chao Ma, Jihuai Zhu, Lelun Wu, Lyumanshan Ye and 5 others
Shanghai Jiao Tong University · Alibaba Group · Shanghai Innovation Institute · GAIR · University of Science and Technology of China
Published September 25, 2026 · on the feed September 28, 2026 · ▲ 6 · arXiv 2609.30837
The abstract
Multi-teacher on-policy distillation (MOPD) integrates specialized capabilities into a single student, but existing practice typically hard-routes each prompt to a domain-matched teacher for the entire rollout. This dependence on prompt-level domain labels restricts using unlabeled training mixtures and leaves complementary signals from other teachers unused. We introduce MOPD-Router, a framework that routes supervision over the full teacher pool at each token, without domain labels or training a separate routing model. Its plug-in interface supports different metrics for selecting and weighting teacher-specific OPD signals. Within this interface, we propose ExpertAlign, which scores each teacher by whether its correction to the student at the current token expresses the specialization that teacher acquired during post-training, and compare it against two reference metrics built on teacher confidence (Entropy) and teacher-student discrepancy (Novelty). Experiments on unlabeled and domain-labeled training mixtures under strong-to-weak and same-size distillation scenarios show that ExpertAlign achieves the strongest overall performance in all four settings. On unlabeled data, it improves the overall score by 5.88 (+12.3%) points over Mean aggregation; on domain-labeled data, it outperforms standard MOPD by 3.95 (+7.8%) points without using available domain labels. These results demonstrate token-level routing can exploit cross-domain complementary supervision, and reduce exclusive reliance on prompt-level domain assignment. Code is available at: https://github.com/TURLEing/MOPD-Router.
The Brief
2:34A short podcast summary of the paper. Written and voiced by AI.
Transcript
MOPD-Router. Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation. By Tianze Xu and twelve others, from Shanghai Jiao Tong University, Alibaba Group, and three other institutions. Published on the twenty-fifth of September. Its code repository is worked out: about nine hundred and fifty source files, with a readme and a licence.
Multi-teacher on-policy distillation, or MOPD, folds the specialized capabilities of several teachers into a single student. In existing practice, each prompt is typically hard-routed to one teacher matched to its domain, and that teacher supervises the entire rollout. The arrangement depends on a domain label for every prompt, and two costs follow. Training mixtures that carry no labels become hard to use. And whatever complementary signal the other teachers could have offered goes unused, because only one of them is ever consulted.
MOPD-Router drops the prompt-level decision. It routes supervision over the full pool of teachers, and it does so at each token rather than once per prompt. It needs no domain labels, and it does not train a separate routing model to make the choice. Instead it exposes a plug-in interface, where different metrics can be used to select and to weight the on-policy distillation signal coming from each teacher.
Within that interface the authors propose a metric called ExpertAlign. At the current token, it scores each teacher by asking whether that teacher's correction to the student expresses the specialization the teacher acquired during post-training. ExpertAlign is compared against two reference metrics that plug into the same interface. One, called Entropy, is built on teacher confidence. The other, called Novelty, is built on the discrepancy between teacher and student.
Experiments cover four settings: unlabeled and domain-labeled training mixtures, each under strong-to-weak and same-size distillation. ExpertAlign achieves the strongest overall performance in all four. On unlabeled data, it improves the overall score by five point eight eight points over Mean aggregation. That is a relative gain of twelve point three percent.
On domain-labeled data the comparison is with standard MOPD, which uses those labels to route. ExpertAlign outperforms it by three point nine five points, or seven point eight percent, without using the domain labels that were available to it.
The results, as the authors read them, show that token-level routing can exploit supervision that is complementary across domains. They also loosen a dependency. Once supervision is chosen token by token from every teacher, prompt-level domain assignment is no longer the only way to decide who teaches, and exclusive reliance on it is reduced. Code for the framework is released.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Three experts, one student, and a rule that lets only one expert speak per question.
You have three specialist teachers, for maths, code and instruction following, all grown from the same base model. You want one student that does all three, trained on its own answers with token-by-token feedback from teachers.
The road ahead
- 1 · The problem ~7 min Merging three specialist teachers into one student is easy only when every prompt carries a domain tag and needs just one teacher.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Route teacher feedback per token over all teachers, weighting each by a plug-in score, best when it checks the teacher's post-training specialization.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min At each token, query every teacher, score its correction, turn scores into weights, and train the student on the weighted feedback.
- 6 · The experiment ~6 min You’ll guess: On domain-labeled data, how did token-level routing with ExpertAlign, which ignores the labels, compare with standard MOPD routing by those labels?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.