Recursive self-improvement of AI research agents

Recursive self-improvement of AI research agents. By Dhruv Srikanth and four others, from Weco AI. Published on the twenty-second of September. No code repository is linked to it.

AI agents are starting to automate research and development across the AI stack, from improving training efficiency to optimizing inference. A natural next step is to improve the research efficiency of the agents themselves. The authors motivate this with a long-standing trend: more cumulative spending on research and development yields diminishing returns. Sustained self-improvement offers a way to push against that. If the agent doing the research also gets better at research as it goes, each round of spending buys an improved researcher as well as a result.

When an AI research agent's own code is what is being optimized, each accepted rewrite becomes the agent that the next round edits. The authors call this loop recursive self-improvement. AIDE squared is a system that implements the loop for a frontier AI research agent.

The loop has three steps. The agent proposes changes to its own code. It benchmarks the modified versions of itself on a suite of AI research and development tasks. And it keeps the changes that perform best on hidden evaluations. The winner then becomes the starting point for the next round. Two details carry weight. Selection is made on hidden evaluations, so a change cannot be kept merely because it looks good on tasks the agent can see.

In an autonomous eight-day run, AIDE squared found seven successive improvements. They ranged from a new search policy to memory mechanisms that compress and manage the agent's growing context. The gains generalize to four held-out benchmarks spanning machine learning engineering, heuristic algorithm engineering, and physics-based weather forecasting. The last of these is out of distribution relative to the tasks used for selection. That is the test that matters most for a self-improvement loop. Changes selected on one set of tasks could easily be tuned to those tasks alone, and holding out whole benchmarks, including one from a different domain, is how the authors check that they are not.

On all four, the strongest discovered agent matches or exceeds a human-engineered production research agent that ranks among the strongest on FML-Bench. And on a separate held-out task family, the discovered agents show less reward hacking, a property the loop never explicitly optimized for. The rate falls from fifty-five percent to thirty-two percent over the run, seven percentage points below the human-engineered agent. The loop was selecting for performance, not for honesty, yet the agents it produced gamed rewards less often as the run went on.

The claim rests on two results. The improvements travelled beyond the tasks that selected them, including to a domain outside their distribution. And a property no one was selecting for, less reward hacking, improved alongside the ones that were. The authors conclude that a research agent can raise its own research efficiency through recursive self-improvement, and that the gains transfer to tasks and domains the loop never saw. The comparison point is what gives the result its scale. The discovered agent was not measured against a weak starting version of itself, but against a production agent engineered by people, and it matched or beat that agent on every held-out benchmark.