Dynamic Harness Search: Building Multi-Agent Systems Per-Query via Prediction

Dynamic Harness Search. Building Multi-Agent Systems Per-Query via Prediction. By Som Sagar and four others, from Google and Arizona State University. Published on the second of October. No code repository is linked to it.

An agent harness specifies the roles, the instructions, the tools and the communication structure a system uses to solve a task. The right harness depends on the query. The trouble is that the value of each design choice is only observable by executing it. Tailoring a harness to each query has therefore meant one of two expensive things: executing alternatives at inference time, or costly manual design. The first pays for every candidate on every query. The second pays in human effort, query after query.

SHIFT moves execution out of the per-query search loop. Instead of running candidates to find out how good they are, it predicts. Two learned pieces do the predicting. A local LLM architect learns, from search, a policy over harness-building actions. And a value function learns from measured executions to predict a utility, one that balances accuracy against execution cost.

When a query arrives, Monte Carlo tree search uses those predictions to construct a harness for it. The architect's policy proposes which harness-building actions to take, and the value function estimates how good the result would be, trading accuracy against cost, without the harness having to be run first. The output is a harness built for this query: its roles, instructions, tools and communication structure. Execution has not disappeared. It has moved to where the value function learns from measurements, rather than sitting inside every search for every query.

The evaluation covers nine thousand one hundred and ninety-three tasks across six benchmarks, ranging from math to document and general-assistant tasks, with a Gemini 3.5 Flash executor. SHIFT attains the highest mean accuracy, about eighty percent. It outperforms seventeen baselines spanning prompting, prompt optimization and workflow search, and exceeds the strongest of them by seven point two percentage points.

A cheaper mode of SHIFT also beats every baseline on mean accuracy, while using thirty-two percent fewer execution tokens than the strongest baseline. Two further results speak to the design. Choosing structure, instructions and tools jointly beats choosing only instructions or only tools, by up to nine point one percentage points. And learned value selection, given pools of candidates, picks harnesses that are more accurate and cheaper to execute.

Underneath the numbers is a trade. Trying every option for every query is expensive, so SHIFT pays instead to learn what trying would have shown. Once that is learned, building a harness for a new query is a search over predictions, not a series of trial runs.