RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Sibo Zhu, Shicheng Fan, Xinyue Wang, Wenyi Wu, Kun Zhou, Biwei Huang
Aether AI · University of California San Diego · University of Illinois Chicago
Published September 14, 2026 · on the feed September 15, 2026 · ▲ 84 · arXiv 2609.15364
The abstract
Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce RSIAgent, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a broad-then-deep exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.
The Brief
2:06A short podcast summary of the paper. Written and voiced by AI.
Transcript
RSIAgent. Autonomous Exploration for Recursive Self-improvement in New Environments. By Sibo Zhu and five others, from Aether AI, the University of California San Diego, and one other institution. Published on the fourteenth of September. Its code repository is worked out: about one hundred and forty source files, with a readme and a licence.
Digital agents often have to adapt to new environments, whose interfaces, tools and failure modes are not fully captured by pretrained models. Whatever the model learned before, the particulars of a new environment have to be learned in that environment. Knowledge that a pretrained model lacks has to come from somewhere, and here it comes from the environment itself.
RSIAgent is a training-free multi-agent framework for recursive self- improvement, and it improves through autonomous memory construction rather than through training. It coordinates three roles: curriculum, actor, and verifier agents. Together they continually explore the environment, validate outcomes, and retain knowledge specific to that environment, including reusable causal relationships between actions, conditions and consequences. One agent sets what to try, one tries it, and one checks whether it worked.
Exploration runs broad first, then deep. Parallel broad recursive self- exploration discovers diverse structures in the environment. Focused deep self- exploration then goes after hard cases, hidden constraints, boundary conditions, and causal dependencies that were previously unknown. The memory that results is frozen, and can be reused directly for downstream tasks without updating model parameters. Breadth maps the environment; depth probes its difficult corners.
Experiments on OSWorld-v2 and Agent's Last Exam show RSIAgent substantially improving strong open-source models. With it, Kimi-K3 and GLM-5.3 outperform frontier closed-source models, GPT-6 among them. The gains are in open-source models, and the comparison is with closed-source frontier models.
Nothing in the model's weights changes. What improves is what the agent knows about a particular place, gathered by exploring it, checked by a verifier, and kept as a frozen memory that later tasks can draw on.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
New software, no manual, and no budget to retrain. Where does the know-how come from?
You drop a capable digital assistant into software it has never seen. Its interfaces, tools and failure modes aren't in what the model learned before, so it stumbles.
The road ahead
- 1 · The problem ~6 min A capable digital assistant stumbles in software it has never seen, and the usual fix of collecting data and retraining is costly and often impossible.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Without any training, the agent explores the new environment itself, checks outcomes, and stores cause-and-effect knowledge in memory, exploring broadly first and deeply second.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Explore broadly in parallel, explore hard cases deeply in sequence, freeze the memory, then reuse it unchanged on real tasks.
- 6 · The experiment ~6 min You’ll guess: With RSIAgent, how did open-source Kimi-K3 and GLM-5.3 compare with frontier closed-source models such as GPT-6?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.