ReGE: Stronger Agents through Experience Synthesis

ReGE. Stronger Agents through Experience Synthesis. By Jiarong Liang and five others, from the University of Waterloo, the University of North Carolina at Chapel Hill, and one other institution. Published on the twenty-fourth of October. Its code repository is public and real: about twenty-five source files.

When an agent makes repeated attempts at an interactive task, each attempt uncovers something. Facts about the task, operations that turned out to be useful, failures that should not be repeated. All of that could inform the next execution, but only if it is turned into something a fresh attempt can actually use. A pile of raw execution records is not yet guidance.

ReGE is a framework for making that conversion: recorded experience in, guidance for a fresh attempt out. Its main implementation, ReGE-Syn, does it with a single coach call. That one call jointly synthesizes bounded execution records and the available artifacts into self-contained guidance, guidance that connects the task-specific findings with proposed actions and with the relevant pitfalls.

The roles are a coach and a student. The student starts from the task's initial environment state, reads the guidance, and adapts it to what it is currently observing. One constraint holds throughout: nothing gets bigger. Generating the source attempts, synthesizing the guidance and executing as the student all use the same base model. There are no parameter updates and no stronger teacher. The model is simply learning from its own earlier tries.

Across four benchmarks, ReGE improves accuracy over unguided execution by an average of nine point nine percentage points for GPT, and eight point eight percentage points for Claude. Cost is measured on matched BrowseComp-Plus subsets, against an adapted version of AggAgent. There, ReGE-Syn cuts the tokens spent constructing guidance by ninety-eight point two percent on GPT and by ninety-five point five percent on Claude, with observed success.

Counting construction and execution together, input plus output, and leaving out source generation, total tokens fall by fifteen point seven percent on GPT and by thirty-five point four percent on Claude. Ablations on BrowseComp-Plus add two more observations. Access to process information is associated with the student taking fewer steps. And joint synthesis produces substantially shorter guidance than either Direct history or Independent summaries, again with observed success.

The authors read these results as showing how agents can use their own recorded experience to improve later task completion. They also point to a further use: a low-cost way to consolidate the execution records of parallel agent systems, such as agent swarms. Many attempts running side by side leave many records behind, and one coach call is offered as the way to fold them into a single set of instructions for the next try.