ParetoRSI-Gym: Can AI Agents Improve Language Models Across Multiple Domains?

ParetoRSI-Gym. Can AI Agents Improve Language Models Across Multiple Domains?. By Haojin Wang and eight others, from the University of Illinois Urbana-Champaign, the University of California, Berkeley, and three other institutions. Published on the twenty-fifth of October. Its code repository is public and real: about twenty source files.

Recursive self-improvement asks an agentic system to design, train and evaluate its own successor. Data-driven post-training is a central piece of that loop. Yet the existing evaluations of it use a single benchmark. An agent picks from a static pool, or distills a stronger model, to push one score up in one run. Real post-training does not look like that. It spans many domains at once, and a gain in one domain has to be weighed against a regression in another.

ParetoRSI-Gym measures whether an agent can improve a language model across several domains together, through data alone. Everything except the data is held fixed: the initial LLM, the toolset, the SFT pipeline, and five domains, namely math, code, finance, medicine and science. Generalization benchmarks in each domain are kept separate from the scores the agent optimizes against, so a real capability gain can be told apart from overfitting to those scores. Holding the rest constant matters. If the model, the tools and the training recipe never change, then whatever difference appears between runs has to come from the data the agent chose, which is exactly the skill being measured.

The agent works in a loop. It searches for data, curates it, trains, reads back the per-domain results, and goes around again. The study runs that loop with three agent systems, on three base models, for five rounds, over multiple seeds. Each round gives the agent fresh per-domain feedback to act on, so it can see where a change helped one domain and hurt another, and decide what to gather next. The name points at the same idea: progress here is a trade-off surface across domains, not a single number.

The strongest system is Claude Code with Opus-5. It raises the generalization scores of its proxy-best checkpoint by nine point two percent relative to the starting model. In absolute terms that is two point three points. Both figures are averaged with equal weight over the three base models and the five domains.

Hidden inside that average is a much larger single-domain gain: up to sixty-three percent relative, for medicine on Ministral-3-3B. Further analysis turns up two more findings. The harness matters as much as the model it drives. And each agent keeps a data selection strategy that is unique to it.

The authors offer the work as a step toward evaluating recursive self-improvement systems in real-world machine learning engineering. Its contribution is a change of question. Pushing one number up in one run is replaced by improving a model across five domains while generalization benchmarks check that the improvement is real. Under that test, the best agent gains a few points on average, with one domain jumping far more, and how the agent is wrapped matters as much as which model sits inside it.