Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine

Terminal-RSI. Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine. By Junyao Yang and eight others, from Tencent Hy Frontier, the National University of Singapore, and three other institutions. Published on the fifteenth of October. No code repository is linked to it.

A terminal agent has two conversions to get right. The outcome of a verifier has to become a policy update that is actually useful, and partial progress on a task has to become complete resolution. Getting either wrong leaves the agent with signal it cannot use or work it cannot finish.

Terminal-RSI alternates two phases. Recursive Self-Rewrite, or RSR, works on trajectories that have been verified and rewrites them. Recursive Policy-Refine, or RPR, refines the policy while inheriting its critic. Each phase hands off to the other, so improvement comes in rounds rather than in a single pass.

Several pieces support the alternation. Turn-Boundary Skip-Observation GAE, written TB-GAE, retains token-specific baselines while applying decay across interaction turns. Two rewards divide the work of credit. Family Dense Reward calibrates semantic progress across task families, and Success-Prioritized Reward separates strict success from bounded partial credit. A Warm-up Critic Model, a recursive critic handoff, and learning rates specific to each stage coordinate the two refinement phases.

The full sequence has five stages: the Qwen3.8-27B base model, then RSR-1, RPR-1, RSR-2, and RPR-2. What comes out is Terminal-RSI-27B. It scores nineteen point eight four percent on Terminal-Bench 4.0, eighty-six point five two percent on Terminal-Bench 2.1, sixteen point nine five percent on Terminal-Bench 3.0, and seventy-nine percent on Terminal-Bench Hard.

On LHTB, process reward rises from zero point two one to zero point three seven, and strict completion goes from zero out of forty-six to five out of forty-six. On SWR100, success doubles, from three percent to six percent.

Read in order, the stages tell a story of their own. Verified trajectory rewriting and critic-inheriting policy refinement are coupled recursively, and each retained stage consolidates behaviors the agent has discovered. The direction of travel is from partial local repairs toward strict terminal verification, which is the second of the two conversions the work set out to make.