FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

FlexRouter. Learning Complementary Model Sets for Flexible LLM Routing. By Wang Wei and eight others, from Virginia Tech, Independent Researcher, and three other institutions. Published on the twenty-ninth of September. No code repository is linked to it.

A language model router decides which models answer a given query. Existing routing methods score each candidate model independently and then take the top k. Two things go wrong with that. Scoring models one at a time ignores the correlations between them, and fixing k enforces a rigid computational budget whatever the query. The practical consequence is redundancy. A router can pick several models that share the same failure modes, so when one is wrong the others tend to be wrong with it, and the chance that anything in the selected set succeeds stays lower than it could be.

FlexRouter treats the selection as a set, and asks for a set whose members complement one another. Its objective is answer coverage: the probability that at least one selected model produces a correct response. That target is chosen to match how inference pipelines are actually used, where several candidate outputs are generated and a downstream verifier, or a user, picks the final one. Under that arrangement, a second model that fails in the same places as the first adds cost without adding coverage.

Routing is formulated as a coverage-oriented subset selection problem, and the routing policy is modelled with Determinantal Point Processes. The authors choose these because they naturally capture two things at once, how competent each model is and how redundant the models are with each other. Training has an obstacle, since there is no ground-truth target subset to imitate. The answer is a training objective built by marginalizing over failure sets, which optimizes coverage directly. At inference time, a greedy strategy adds models according to their marginal gains in log-determinant. Because the procedure stops on its own, the router adaptively decides how many models to use rather than working to a predefined budget.

Experiments run on the large-scale RouterEval benchmark. Against strong baselines, FlexRouter achieves higher coverage with lower redundancy, on in-domain tasks and on out-of-domain tasks alike, while keeping inference cost flexible.

Two assumptions are retired together here. One is that the best set of models is simply the best individual models stacked up. The other is that the set should always be the same size. Choosing for complementarity, and letting the gains decide when to stop, replaces both.