DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck

DuoDiscover. Scaling Autonomous Discovery Beyond the Evaluation Bottleneck. By Shubham Agarwal and twenty-three others, from Google, UC Berkeley, and one other institution. Published on the twelfth of October. No code repository is linked to it.

Evaluation has driven much of the progress in AI, and autonomous discovery depends on it just as heavily. The trouble starts when discovery is pointed at real engineering work. There, one true evaluation, a full benchmark run or a serving trace, can take hours. A search loop that needs feedback to make progress can then afford only a handful of true evaluations in total. The bottleneck is not ideas, and not the agents proposing them. It is the price of finding out whether any of them helped.

DuoDiscover is a search system that treats the evaluator as something to be discovered alongside the solution. It works out what to optimize and, at the same time, how to measure progress on it cheaply. The abstract names three mechanisms that make this work, and each answers a different risk.

The first is the yardstick. Every worker in a fleet of coding agents co-discovers two things: an optimization target, and a fast proxy evaluator tailored to it. The agent then hill-climbs that yardstick instead of the slow true evaluation.

A cheap proxy invites a familiar failure, which the second mechanism is there to catch. Search can exploit any gap between a yardstick and the real objective, so fidelity tests keep rechecking how each yardstick ranks things, and any yardstick that drifts is invalidated. Third, the fleet is organised from the bottom up. Nobody decomposes the problem centrally. A lightweight orchestrator spawns and terminates agents, briefs the ones already running, and seeds new agents with work that others have confirmed. The authors describe the result as autonomous optimization that reaches from software all the way to chip architecture.

Testing covers two repository-level benchmarks and four GPU and TPU kernel suites, together spanning hundreds of kernels. On SWE-fficiency, DuoDiscover doubles the average speedup achieved by human experts. On SOL-Exec, it ranks first on one hundred twenty-one of two hundred thirty-five kernels, and lands in the top three on two hundred twelve.

Then come two engines described as heavily optimized already. On vLLM the gain over the base code is one point nine two times. On SQLite it is one point three eight times. The hardest case is a CPU microarchitecture task where a single evaluation can take up to twenty-eight hours. There DuoDiscover surpasses both the best human design and ArchAgent v2, a system built on AlphaEvolve. It reaches one point six five times the gain that ArchAgent v2 achieved, and gets there in sixteen point seven days, against more than forty days for the earlier system.

Read as a whole, the system is a wager about where the expense in discovery lives. When each honest measurement costs hours, the search spends its effort on cheap measurements of its own making, and then spends a little more keeping those measurements honest. The chip design result is the clearest test of that wager, since it is exactly the setting where true evaluations are rarest, and the authors present it as moving the frontier of autonomous chip design forward.