Heterogeneous Agentic Coevolution

Heterogeneous Agentic Coevolution. By Wenqian Ye and sixteen others, from Meta AI and the University of Virginia. Published on the twelfth of October. No code repository is linked to it.

Self-improving language model agents have shown that AI systems can drive their own intelligence forward. Many domain-specific applications, though, run on specialized foundation models. These have strong domain expertise but still depend on improvements that humans design. Extending recursive self-improvement to them runs into a moving-target problem. As the specialist gets better, the strategies used to advance it also have to change. Whatever improved it last month may not be what improves it now.

Heterogeneous Agentic CoEvolution, or HACE, evolves two different kinds of model together: the specialized foundation model, and a language model agent. They improve through reciprocal learning, each adapting to the other's progress. The word heterogeneous is doing work here. This is not a model improving itself, but two unlike models improving each other.

Each side learns in its own way. The specialist improves through supervised learning. The agent is trained with on-policy reinforced self-training, guided by gains that have been verified on the current specialist. Then the loop closes. The agent's exploration informs the next round of specialist training, so both keep adapting to where the other has got to. The agent is rewarded for changes that measurably help the specialist as it stands now, and the specialist learns from what the agent went looking for. Neither side is trained against a fixed partner. Each round, the partner is a little different from the last.

One more layer keeps that reciprocity going. HACE uses evolutionary search to optimize the interaction harness between the two models. That harness governs how they exchange information and how training experiences are selected, and it too is adapted as their capabilities evolve. So there are three things changing at once: the specialist, the agent, and the way the two talk to each other. The last of these is what sustains the reciprocal learning, by letting the terms of the exchange shift as both partners grow.

The evaluation covers three challenging domains that sit far apart: recommendation systems, protein function prediction in biology, and crystal stability prediction in material discovery. In all three, HACE outperforms the evaluated self-improving methods on the primary task metrics. The abstract reports the comparison qualitatively and gives no figures for it.

The authors present heterogeneous coevolution as a universal approach to improving specialized foundation models, through reciprocal interaction with a learning agent. The breadth of the test domains is what that word universal leans on. The underlying move answers the moving-target problem directly: rather than fixing the improvement strategy, the strategy is itself a learner that keeps pace with the model it is improving.