PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Plugins

PluginRSI. Recursive Improvement of Agent Harnesses with Reusable Plugins. By Yaorui Shi and seven others, from the University of Science and Technology of China, Meituan, and two other institutions. Published on the twenty-sixth of September. It links to a code repository that is no longer there.

Around every language model agent sits a harness, and the harness is a central determinant of how well the agent performs. Recent methods improve harnesses by searching over complete programs. The difficulty is granularity. When the unit of search is a whole program, the individual mechanisms inside it are hard to isolate, and hard to carry over into the next harness.

PluginRSI changes the unit. A harness is represented as a composition of atomized plugins, and harness evolution is organised around those plugins rather than around whole programs.

Each plugin is improved independently. Improved plugins accumulate in a shared library, and at every iteration plugins from that library are recombined into new harnesses. The library is what carries across iterations: a mechanism that worked once stays available to be used again.

Against existing harness optimization methods, PluginRSI improves results on three kinds of task: software engineering, command-line interaction, and question answering. The harnesses it produces keep their advantage when transferred to other solver models, with no further optimization. And the evolved plugin library speeds up later optimization that starts from the initial harness, helping convergence come faster and reach higher on unseen tasks.

Where earlier searches produced a finished program, this one also leaves behind a growing set of parts. The authors read their results as showing that accumulating reusable mechanisms is an effective basis for continued harness improvement, and the transfer and acceleration results are what carry that case.