Harness-Zero: Harness Distillation via Agent-as-Harness

Harness-Zero. Harness Distillation via Agent-as-Harness. By Haoran Ye and four others, from Peking University, Google, and one other institution. Published on the twenty-first of September. Its code repository is worked out: about five thousand five hundred source files, with a readme and a licence.

An agent harness is the external system that mediates between a model and its environment, and a good one can improve an agent's performance substantially. The catch is that the gains belong to the harness, and only appear while that harness is in place at deployment. The best harness also differs by domain, by instance and by model. So a general-purpose agent faces an awkward choice: settle for one shared harness that is suboptimal, or route between an ever-growing collection of specialized ones.

The alternative studied here is harness distillation. Use a harness optimized for a domain or an instance as guidance during training, and move the behaviors it induces into the model's weights, so that the gains survive under one fixed target harness. The difficulty is that the two harnesses do not match. They differ in their action space and in what information is available. Guidance from the optimized harness cannot simply be used as supervision for the target one.

Harness-Zero makes the transfer through what the authors call agent-as-harness. A harnessing agent, guided by the optimized harness, corrects the student's responses before they are executed, and it makes those corrections in the target harness's action space. That converts the optimized harness's guidance into training demonstrations the target harness can actually use.

Fine-tuning on the resulting trajectories internalizes the behavior the harness had been inducing. Once it lives in the model, the specialized harness can be removed at deployment. The sequence, then, runs in three steps. The student acts in the target harness. The harnessing agent, informed by the optimized harness, corrects those actions before they run. And the corrected trajectories become fine-tuning data, so the student learns to act as if the optimized harness were still guiding it.

Experiments cover knowledge work, tool use and science domains, and report three findings. First, for frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. Delivering the harness's guidance through an agent, rather than through code, is the better route on that comparison.

Second, with the specialized harness removed at deployment, Harness-Zero lifts the base model's macro-average task success from twenty-three point three percent to forty-four point three percent. That is higher than the forty-one point seven percent the model reaches with the harness still attached.

Third, Harness-Zero recovers behaviors the harness induced that were absent from the base model, with eighty-two point three percent average recovery across twenty-eight patterns in the three domains.

Of the three, the second finding carries the argument. The goal was to keep a harness's gains without keeping the harness, and the distilled model, running without it, ends up ahead of the base model running with it. What began as scaffolding around the model has been absorbed into it, and that sidesteps the choice between one shared harness and many specialized ones: deployment needs only the single fixed target harness.