MILO: Automated Harness Discovery via Orchestrated Multi-Agent Evolution

MILO. Automated Harness Discovery via Orchestrated Multi-Agent Evolution. By Prithwish Jana and eleven others, from the Georgia Institute of Technology, AWS AI Labs, and two other institutions. Published on the twenty-ninth of September. No code repository is linked to it.

A modern agentic system is two things: an AI model, and a harness around it that controls execution and how the agent interacts with its environment. The harness design strongly affects how the system performs over long horizons. But the space of possible harnesses is combinatorial, searching it takes substantial human effort, and that effort has to be repeated every time the models change. Automated methods exist, yet they explore narrowly. Some optimize only one component, such as prompts or skills. Others get trapped by search strategies that are fixed and exploitative.

MILO, short for Meta-evolutionary Island Orchestration, evolves two things at once: the agent harnesses, and the strategy used to discover them. The search is not a fixed procedure applied to the harness. It is itself something that changes as the search goes on.

Three parts make that work. The first is a hierarchical lineage memory kept over island-based trees, which records rejected mutations as negative evidence rather than discarding them. The second is a set of mutator agents, one per island, that rewrite complete harnesses, drawing on the global search history and on feedback specific to the parent being changed. The third is an orchestrator that adapts the search itself, through lineage grafting and speciation, by reassigning mutators, and by revising the curriculum.

The evaluation covers Terminal-Bench two point one, PaperBench and DeepSWE, with both a frontier model, Opus four point eight, and an open-weight one, gpt-oss with one hundred and twenty billion parameters. Harnesses discovered by MILO outperform eight state-of-the-art harnesses and six search methods. With Opus, MILO improves resolution over its initial harness by twelve percent, twenty-eight point three percent and ten point three percent on the three benchmarks. The best prior search methods gained four point five percent, eighteen point three percent, and nothing.

On Terminal-Bench alone, MILO reaches eighty-six point one percent, plus or minus two. The official leaderboard's top entry stands at eighty-three point eight percent, plus or minus two point three. MILO gets there using twenty-six percent fewer tokens than its initial harness.

The method also leaves agent benchmarks behind. On open problems from EinsteinArena, MILO improves the best-known upper bounds for the Erdos minimum overlap problem, and for the first and third autocorrelation inequalities. The changes live far down the decimal places. On the third inequality, for instance, the bound moves from one point four five zero eight one to one point four four eight eight nine.

Two choices separate MILO from the methods it is measured against. It rewrites whole harnesses instead of tuning one part, and it lets the search strategy evolve alongside the thing being searched for, keeping even its failures as evidence. The same machinery that passes the top Terminal-Bench leaderboard entry also nudges down bounds on open mathematical problems.