Self-Evolving Coding Agents via Harness-Policy Optimization

Self-Evolving Coding Agents via Harness-Policy Optimization. By Zhepei Wei and seven others, from Microsoft Research, the University of Virginia, and one other institution. Published on the sixteenth of October. Its code repository is worked out: about two hundred and fifty source files, with a readme and a licence.

A coding agent's performance depends jointly on two things: the policy model, and the harness that mediates its interaction with the environment. The authors argue the two are mutually reinforcing, and that evolving only one of them is suboptimal. The failure runs both ways. A harness tuned for one checkpoint may not target the failure modes of an updated policy. Policy learning, in turn, is limited to the trajectories that a fixed harness elicits. Improve either side alone and the other side holds it back.

Self-HPO is a recursive self-improving framework for coding agents that alternates between optimizing the harness and optimizing the policy. Its distinctive move is who proposes the harness: the evolving policy itself. The model being trained is also the one redesigning its own surroundings, so harness and policy are no longer tuned by separate processes on separate schedules.

Each round has two steps. In the H-step, the current policy analyzes its own failures and proposes new harness candidates. In the P-step, the policy is updated by reinforcement learning under the selected harness. Then the loop closes: hard tasks identified during training become the next harness search set. That keeps harness optimization aimed at the failure modes the policy has now, rather than the ones it had when the harness was last tuned.

Across three coding benchmarks, Self-HPO consistently outperforms both harness-only and policy-only baselines. For Qwen3.5-35B-A3B, the average resolve rate rises from fifty-seven point eight percent to sixty-four point eight percent, and the gains generalize to out-of-distribution settings.

Further analysis separates the two components. Each optimized component keeps its gains even when paired with its base counterpart. And their joint gain exceeds the sum of their individual gains, which is the evidence for the claim that they reinforce each other. Self-HPO also improves reliability, measured as pass at one, and expands the capability frontier, measured as pass at k, over the base agent. Finally, with a stronger external model, such as Opus four point eight, serving as the harness proposer, the same framework yields a further performance boost.

The structure is an alternation between two players who are the same model. Harness search is pointed at what the policy currently fails on, and the policy is trained inside the harness that search produced. A combined gain larger than the two separate gains added together is what makes this more than two methods run back to back. The external proposer result adds that the loop is not bound to the policy proposing for itself; that is simply its self-contained default.