EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

EdgeWise. Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation. By Zeru Shi and twelve others, from NVIDIA and Rutgers University. Published on the thirteenth of October. No code repository is linked to it.

Complex, long-horizon agent tasks tend to lean on frontier models running in the cloud, because that is where the strong performance is. Calling such a model at every step of a long trajectory is expensive, and not every step necessarily needs that much capability. The authors test that intuition first. They find that handing the large language model only a small subset of key steps can match the performance of running it throughout, at a significantly lower inference cost.

EdgeWise is built on that observation. It is an edge-cloud collaboration framework in which a small language model, running on the local device, acts as the primary solver. When a step needs more capacity than it has, the small model selectively delegates that critical step to a large model in the cloud.

The delegation has to be decided somehow, and the authors want that decision made without bringing in extra models at inference time. So they introduce a step-level reinforcement learning method that optimizes task solving and delegation jointly, with fine-grained credit assignment for each individual delegation decision. The small model learns both to solve and to know when to ask.

Separately, they introduce a delegation strategy based on an external judge, and use it as an instrument to investigate the design choices in edge-cloud collaboration systematically. Those choices are when to delegate, how the delegation decision should be made, what context should be shared with the cloud, and how often the cloud model should be invoked at all.

In experiments, EdgeWise matches or outperforms running the large model alone, while cutting inference cost by up to sixty-two percent on SWE-Rebench and up to thirty-four percent on Terminal-Bench 2.1. The authors also deploy it on a local workstation and report comparable cost savings in a practical edge-cloud setting, beyond the experimental one.

Everything here rests on one premise: capability is needed unevenly along a trajectory. Most steps of a long task can be handled by a small model sitting on the device. A few cannot. If only those few are hard, the expensive cloud model only needs to see those few, and the bill shrinks accordingly while the outcome holds. The measurement at the start establishes that such a split exists. The reinforcement learning method is what lets the small model find the split on its own, step by step, with no separate router running beside it. And the judge-based strategy turns the remaining questions, about timing, context and frequency, into things that can be studied directly rather than guessed at. What the paper offers, in the end, is a way to keep the strong model in the loop while paying for it only where it matters.