Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs

Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs. By Yuanyi Wang and nine others, from the Hong Kong Polytechnic University and InfiX A I. Published on the twenty-sixth of September. Its code repository is public and real: about twenty source files.

Model merging combines specialised large language models efficiently, with no joint retraining. In Mixture-of-Experts models, though, merging can substantially change which experts get chosen for which tokens. That routing drift is often read as routing failure. The paper asks whether that reading is justified: does drift after merging actually indicate failure, and what evidence should be required before anyone repairs it? Its answer, stated in the title, is that drift on its own is not enough. A correction informed by the source models has to be judged by its effect on the task.

The study covers DeepSeekMoE, OLMoE, and Qwen3-MoE, and comes with a routing analysis toolkit for controlled counterfactual interventions and token-level analysis. By crossing the router inputs and router parameters of the source and merged models, the authors attribute most expert reassignments to shifts in the inputs, not to changed parameters at the same layer. Then comes the central observation. How far routing differs from the source model is a poor predictor of how much next-token likelihood improves when the source route is restored. Different expert selections can also produce mixture outputs that point in similar directions.

So routing failure is given an operational definition: task loss that can be recovered under a specified routing intervention, with every non-routing parameter held fixed. Under deliberate router corruption, these tests do detect recoverable loss. In the merged models evaluated, restoring source routes does not establish reliable task benefits. As a case study the authors propose Selective Router Repair, or SRR, and find that the token-likelihood advantages of source specialists do not reliably pick out which local corrections will help.

The conclusions are tied to the three model families studied and to the merged models evaluated within them. The negative result about restoration is a statement about those models, set against a positive control: when the router is deliberately broken, the tests catch it. Selective Router Repair is offered as a case study, not as a general remedy. The toolkit and the SRR code are released.

Watching routes change and calling it breakage turns out to skip a step. A different expert can land in much the same place, and most reassignments trace back to shifted inputs rather than a corrupted router. The standard the paper sets is simple to state: show that an intervention recovers task loss, and only then call what it fixed a failure.