SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles

SkillForge. Co-Evolving Skills and Agents via Dynamic Skill Lifecycles. By Yuyao Ge and seven others, from the Institute of Computing Technology of the Chinese Academy of Sciences, the University of California, Merced, and one other institution. Published on the seventh of October.

Memory-augmented reinforcement learning helps language model agents with complex, long-horizon tasks, and skills are one form that memory takes. A skill pairs an instruction with a condition saying which task types it applies to: do this, when facing that kind of task. The difficulty is what happens over time. The policy is not standing still while it learns. As it improves, keeping every skill indiscriminately lets obsolete or harmful entries pile up, and those entries can mislead the very agent that once benefited from them. A library that only ever grows ends up holding advice written for a weaker version of the agent.

SkillForge treats the skill library as something with a life cycle. It is an agentic reinforcement learning method in which each skill moves through four states: trial, active, stable, and retired. Movement between those states is driven by fitness. The method compiles the library and keeps evolving it, so that the skills and the model co-evolve throughout training, rather than one being fixed while the other learns.

Training has a preparatory phase and a main phase. Before reinforcement learning begins, an evaluation phase uses the base model's own rollouts to retire low-fitness skills in advance. The model is, in effect, the first judge of which skills deserve to be kept. The filtered library that remains then seeds supervised fine-tuning. Reinforcement learning takes over from that checkpoint, and at every iteration three processes keep reshaping the library alongside policy optimisation: selective retirement, stabilisation, and mutation guided by a language model. Skills are forged and discarded while the policy is still being trained.

Across multiple interactive agent benchmarks, SkillForge achieves the highest aggregate success rate, with up to seven point eight percent relative improvement over the strongest baseline, and it keeps the skill library compact throughout training. The authors also introduce SkillFurnace, a dataset of more than five thousand annotated records. It bundles SFT trajectories filtered by retirement, evolved skill libraries with fitness annotations, and retirement events labelled by humans with failure categories, to support research on skill quality and lifecycle management.

The premise is that a skill library should be able to forget. Retiring skills that no longer help, before training and throughout it, is how SkillForge lets the library keep pace with a policy that is still changing, and the compact library at the end is the visible sign that forgetting is happening, rather than a store of stale advice growing in the background.