ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

ModularRSI. Modular and Generalizable Recursive Harness Self-Improvement. By Siwei Wu and thirteen others, from Beihang University, the University of Manchester, and four other institutions. Published on the fourteenth of September. Its code repository is worked out: about one thousand five hundred source files, with a readme and a licence.

Recursive self-improvement, or RSI, has recently been extended to agent harnesses for long-horizon coding and terminal tasks. The idea is that an agent can improve its own execution mechanisms from experience. Making those improvements generalize is hard, and the paper names three reasons.

First, evolving a harness on an evaluation benchmark, or on part of one, makes it hard to tell reusable improvements from adaptation to that particular benchmark. Second, updating from a single trajectory can confuse a systematic harness deficiency with the reasoning and solution details of one instance, and the resulting changes transfer poorly to unseen tasks. Third, inside a monolithic harness, recurring behavioral deficiencies are hard to localise. Optimizing the whole harness at once entangles unrelated mechanisms, which makes it hard to attribute a change to its cause or to validate it.

ModularRSI answers each of those in turn. It is benchmark-disjoint, contrastive, and modular. The tasks it evolves on never come from the evaluation benchmarks. Evidence comes from contrasting successful and failed trajectories on the same task, and then aggregating across many tasks, so that what surfaces is a deficiency that recurs rather than an accident of one attempt. And the harness is split into parts that evolve separately.

There are five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module evolves independently within a restricted scope of modification, so a change meant for one cannot leak into the others. An integration stage then combines the evolved modules into a unified harness and resolves any conflicts between them. To keep evolution disjoint from evaluation, the authors curate two thousand executable evolution tasks from external sources that do not overlap the downstream benchmarks.

Experiments on TB2.0 and SWE-Bench Verified show consistent improvements on unseen tasks, both in-domain and cross-domain. Since none of the evolution tasks came from these benchmarks, those gains cannot be explained as the harness having seen them before. The evolved harness also transfers across different foundation models, so what was learned belongs to the harness and not to the particular model it was evolved with.

Each design choice maps onto one of the three failure modes. Separate data guards against overfitting to the benchmark. Contrast guards against learning from one instance's quirks. And modules keep each change attributable to the part of the harness it touched, which is what makes it possible to check.