Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling

Self-Credit. Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling. By Fangcong Yin and nine others, from Meta, New York University, and one other institution. Published on the twentieth of October. No code repository is linked to it.

LLM agents are commonly trained with RLVR, and the rewards it typically uses are sparse: a single outcome reward rather than feedback on each step. That makes credit assignment hard, and hardest of all on long-horizon tasks, where many turns separate the decisive action from the reward that eventually judges it.

Self-Credit is an end-to-end agentic RL method that manufactures dense supervision out of sparse outcome rewards. The credit is generated by the policy model itself, as it works, and is then refined by comparing rollouts within a group, the mechanism the title calls intra-group rescaling.

It builds on group-based RL with three components. First, while rolling out, the policy writes summaries that cite earlier turns holding critical evidence. Those citations are the self-generated credit. Second, intra-group rescaling refines that credit by comparing which actions were cited in successful and in failed rollouts for the same prompt. Agreement and disagreement across the group are used to strip away undeserved credit and to further emphasise credit on critical actions.

Third, advantage reweighting closes the circuit. The rescaled credit weights are used to update the advantages of the cited turns, so that the turns the policy pointed to, and the group confirmed, carry more of the learning signal than the turns around them. That is how a sparse outcome reward becomes the denser signal the method set out to create.

On BrowseComp-Plus, Self-Credit improves over the base models by thirty-seven points on Qwen3 at eight billion parameters, and by thirty-six point two points at thirty-two billion. Against the strongest agentic RL baselines, the margins are eight point five and five point six points.

Beyond that benchmark, it improves over the strongest baselines by four point nine points on GAIA and four point seven points on xBench-DeepSearch. Analyses show that the self-generated credit is a useful signal for picking out critical actions, and that intra-group rescaling matters across model scales, especially when applied to failed rollouts.

The policy is asked to point at the turns that mattered, and the group decides how far to believe it. The model's own account of its work becomes supervision only after it has been checked against rollouts that succeeded and rollouts that failed on the same prompt.