Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

Co-Evolving Harnesses and Models. On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails. By Zhou Yu and ten others, from Salesforce A I. Published on the eighth of September. No code repository is linked to it.

An agent harness is everything built around a model: the system prompt, the tool set, the execution hooks, the scaffolding that manages context. Harnesses strongly determine whether agentic tasks succeed, and evolving them automatically can let smaller models do well on domain-specific tasks at a fraction of the cost of a frontier model. Since both harness and weights shape behaviour, the authors ask how harness evolution should be combined with lightweight fine-tuning.

The finding is that the obvious combination backfires. Training a weaker model to imitate a stronger expert, inside a harness evolved for the weaker model, makes it worse. Correcting it on its own rollouts instead works. The difference between the two lies in whose plan the model ends up following.

Across seven enterprise agent tasks, a harness was first evolved with the weaker model. A stronger expert then often used that harness more effectively, which suggested expert supervision might close the gap. But training the weaker model on the expert's complete trajectories under the evolved harness made performance regress on all seven tasks, by four to thirty points, across Qwen3-Coder and Gemma 4. The same procedure helps under the unevolved harness.

The analysis explains why. Imitation does transfer knowledge and increases use of the scaffold, but it breaks the fit between model and harness. The weaker model picks up the expert's planning strategy without the competence to carry it out, and it no longer matches a harness that was evolved around its own native planning style. The harness had been shaped to suit one way of working, and imitation replaced that way of working with another the model could not carry off.

The fix is an on-policy expert-correction pipeline, automated by a meta-level MLE agent. It finds the turn where the weaker model's own rollout goes wrong, and asks the expert to rewrite only that turn. Everything else in the rollout stays the model's own. This keeps the model's planning style intact, and combines the gains of harness evolution with those of adapting the model.

The evidence covers seven enterprise tasks and two model families, Qwen3-Coder and Gemma 4. The regression under imitation is specific to the evolved harness, since the same imitation helps without it. That condition is the heart of the result: the conflict is not between imitation and learning in general, but between imitation and a harness already fitted to how the weaker model plans.

Harnesses and models can evolve together, but only if updates to the weights respect what the harness was built around. Copying an expert wholesale overwrites the planning style the harness depends on. Correcting one failing turn at a time leaves that style in place, and the authors present this as a compatibility-preserving recipe for economical co-evolution on domain-specific enterprise tasks.