AgentBaton: Efficient Model Collaboration in Agentic Tasks

AgentBaton. Efficient Model Collaboration in Agentic Tasks. By Junlong Li and eight others, from NVIDIA, the Hong Kong University of Science and Technology, and one other institution. Published on the twenty-fifth of October.

Long-horizon agentic tasks ask a language agent to interact with its environment over many steps, and the reliable performance tends to come from capable models served by cloud providers, which are expensive. At the same time, a second tier of models is emerging: lower-tier cloud models, and smaller open-weights models that can run on local hardware. They perform worse, but they cost much less. The usual way to mix the two is query-level routing, where each whole request goes to one model or the other. A less explored question sits inside a single task. Could a capable model and an efficient model share one long-horizon job, cutting the cost substantially while keeping the task accuracy where it was?

The authors approach this through model handoffs, where one model takes over from another partway through, and they study the design choices systematically. Two of those choices get the focus: the direction of the handoff, and how the context is transferred from one model to the next. The direction turns out to matter a great deal. Handing off from an efficient model to a capable one generally underperforms simply running the capable model throughout, and buys limited cost reduction, sometimes none at all. The other direction behaves very differently. When a capable model hands off to an efficient one at a suitable stage of task progress, accuracy is largely preserved, sometimes even improved, and the cost falls substantially.

That pattern suggests something modest is enough. A single handoff, done once, kept simple, and timed well, is good enough for cost-effective collaboration. The remaining question is when. AgentBaton answers it with a lightweight detector that learns when to hand off, drawing on recurring patterns in how a task progresses. The capable model opens the task, and the detector decides the moment at which the efficient model can take the baton for the rest of it.

The evaluation spans eleven model pairs and two representative benchmarks. Averaged across them, AgentBaton reduces API costs by sixty-five point nine percent while largely preserving task accuracy.

Relative to capable-only execution, the accuracy gains reach as high as eight point eight percentage points. Further experiments vary the reasoning-effort setting, and across those settings AgentBaton extends the empirical frontier of accuracy against cost, the Pareto frontier, rather than simply trading one for the other at a single point.

The result reorganises what a hybrid agent should look like. The instinct might be to start cheap and escalate when things get hard, and the study finds that this direction mostly fails to pay off. Starting with the capable model and stepping down once the task has progressed far enough is what saves money without giving up accuracy. Once that is clear, the engineering left is small: one handoff, and a detector that knows when to make it.