Evolving to Match Capabilities: Improving Harness Self-Evolution via Skill Adaptation

Evolving to Match Capabilities. Improving Harness Self-Evolution via Skill Adaptation. By Zhihao Yang and nine others, from the University of Chinese Academy of Sciences, Independent Researcher, and four other institutions. Published on the tenth of October. Its code repository is public and real: about twenty source files.

Harness self-evolution offers LLM agents a practical way to improve from execution experience, without updating model parameters. Its effectiveness depends on whether a model can successfully act on the guidance its harness provides. The authors find that harness benefits are not uniform across model capabilities. Skills that improve stronger models can give weaker ones little benefit, or even degrade their performance.

The numbers make the split stark. Across seven models and three benchmarks, the authors identify one hundred and three task-skill pairs where skills improve GPT-5.6-luna by thirty-one point one points, but degrade Gemma-4-E4B by eight point eight points. And in eighty-six percent of failed weak-model runs, the requirement that was violated is already stated in the skill. The guidance was there. It was not carried out.

That motivates skill adaptation: reformulating existing skills to match the capabilities of the executor, so that their guidance becomes easier to execute. The skill is not replaced. It is rewritten for whoever has to follow it. If the weak model fails even though the requirement is written down, then the problem is not missing knowledge in the skill. It is the way that knowledge is put, and that is something a rewrite can change.

To do this, the authors diagnose failures in four areas: tool execution, context management, instruction following and verification. They distill those diagnoses into a fixed meta-skill, which guides adaptation during harness self-evolution. The adaptation is then built into a five-round self-evolution loop, in which each model revises its own skill from its execution trajectories and from verifier feedback. The meta-skill stays fixed throughout. What changes from round to round is the skill being adapted, and the experience each model gathers while trying to follow it.

Guided by the meta-skill, a capable external rewriter, Kimi-K3, adapts skills without any execution evidence. That alone improves the three weak executors by five point one to fourteen point one points over the original skills. Inside the self-evolution loop, across seven executors ranging from Gemma-4-E4B to Kimi-K2.5, the meta-skill consistently improves final scores by two to five point one points over the same loop run without it. The first result shows that adaptation helps even before any execution happens. The second shows that it still adds something once models are learning from their own runs.

On this evidence, a skill is not good or bad in itself. Its value depends on who executes it, and the same guidance that lifts a strong model can sink a weak one. Rewriting skills to fit the executor, with a fixed meta-skill steering the rewrite, is what the gains come from. The authors present their findings as practical design guidance for more effective harness self-evolution methods.