ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis

ExpVoyager. Direct Experience Navigation for Dynamic Agent Skill Synthesis. By Kwangwook Seo and one other, from Yonsei University. Published on the twenty-sixth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

Language model agents that learn from experience have become a key route to self-evolving systems, ones that keep expanding what they can do. Inside that route, synthesizing agent skills has emerged as a promising way to turn accumulated experience into reusable procedural knowledge, a layer of the harness that supplies an agent at runtime. The trouble lies in timing. Existing approaches mostly abstract past experience into fixed procedural knowledge before anyone knows what later tasks will demand. That early compression cuts both ways. It can throw away knowledge that turns out to be critical, and it can keep instance-specific details that no future task will ever need.

The authors reframe skill synthesis as a navigation problem. Instead of distilling experience once and in advance, the agent goes back to what it has accumulated when a task arrives, and explores those past trajectories on demand. Access is targeted and fine-grained: the agent reaches for the particular pieces of experience that the current task calls for, rather than relying on a summary written before the task existed.

ExpVoyager puts this into a framework built around a skill curator. The curator navigates raw experience, not a digest of it, and it can look at that experience across different views and at different resolutions. As it moves, it does two things at once. It keeps identifying reusable procedural knowledge from whatever it is observing. And it keeps track of the knowledge needs that remain unmet, which in turn decide where it should navigate next. Skill synthesis becomes a loop of looking, extracting, and deciding what is still missing, driven by the task at hand.

Extensive experiments are reported to show both effectiveness and versatility. Downstream task performance improves consistently. Gains continue as the experience space scales up, so a larger store of past trajectories keeps paying off rather than saturating. And the framework is shown to be practically compatible with skills that already exist, while accessing experience efficiently.

Much of this rests on one change of timing. Earlier skill synthesis decided what to keep before the demand was known. ExpVoyager keeps the raw record and lets each new task decide what to pull from it, with a curator that knows what it still lacks. Whether experience is a library to summarise or a territory to explore is the choice the paper makes, and it chooses the territory.