Self-Evaluating Recursive Agents

Self-Evaluating Recursive Agents. By TianYi Lyu and nine others, from Tongji University, Independent Researcher, and four other institutions. Published on the fourth of October. Its code repository is worked out: about two hundred and fifty source files, with a readme.

A recursive language-model agent breaks a task apart and hands the pieces to child instances of the same policy. Those children may break their pieces apart again, and the result is a tree of work. Training such an agent is hard for a specific reason. The final outcome can be verified, but the intermediate subtasks are invented by the agent itself, there are many of them, and none of them comes with ground truth. Existing methods score each node with a verifier or a judge. That is costly at scale, and it is blind to whether the decomposition itself was any good.

The authors argue that a recursive agent has to learn three coupled capabilities inside one set of weights: decomposing problems into subtasks, solving those subtasks, and evaluating the outcomes. Each needs its own training signal. Decomposition has to be rewarded for splitting well, solving has to be rewarded for getting the piece right, and evaluation has to be rewarded for telling the difference. SERA, short for Self-Evaluating Recursive Agents, makes the third capability, evaluation, something the policy learns for itself rather than something an outside judge supplies at every node. The question it poses is whether an agent can be taught to grade its own homework well enough that the grades become a usable training signal.

Before delegating, the parent writes a rubric for each child subtask: a set of weighted success criteria. A ranking objective against verified outcomes then trains the rubric generation, so that the criteria come to track genuine subtask success. Alongside that, a complementary leaf-coverage signal gives direct credit for how the task was decomposed.

The division of labour with the judge is deliberate. The judge is consulted only to train the rubric generator. Solving is trained against the agent's own rubric scores, and those scores outperform using the judge directly. So external supervision shrinks as well. Once trained, the rubric has a second use: at inference time it acts as a selector for tree search.

The central finding is about what drives the gains. It is training the policy to generate aligned rubrics. Because the same weights both evaluate and execute, learning to judge subtasks sharpens the agent's ability to solve them. The benchmarks are TextCraft-Synth and TextWorld-Sync. Against strong recursive-agent baselines, SERA improves by five point three eight points on average on the first.

On TextWorld-Sync the average improvement is thirteen point one four points. Using the rubric to guide tree search at inference adds a further two point four three points there, on top of the trained agent. That last number reuses what training already produced: it is the same learned rubric, put to work picking among branches instead of scoring them for a gradient.

What makes the design hang together is that one model holds every role. The parent that writes the rubric is the same policy as the child that is scored by it, so a better sense of what counts as success for a subtask is, at the same time, a better sense of how to achieve it. The judge stops being a permanent fixture at every node and becomes a teacher for one skill, which the agent then carries on its own.