DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

DeLoop. Repeated Actions as Self-Supervision for Long-Horizon Agents. By Tianle Chen and ten others, from Google, Boston University, and three other institutions. Published on the fourteenth of October. Its code repository is public but slight: one source file.

Language-model agents running long tasks often redo work they have already done. They re-issue a tool call made earlier, and get back exactly the same observation as before: a step that adds no new information at all. The authors call this failure a loop. Loops appear natively in Qwen, Llama and Gemini agents, and they become far more common once the agent's history is compressed to fit its context window, which is precisely the situation a long task creates.

DeLoop is a self-supervised training method for reducing loops. It rests on a simple observation: the agent's own rollout already contains the supervision needed to learn from its loops. No outside judge is required to say that a repeated call with a repeated result was wasted. The repetition is visible in the record of the run, and so is what the agent eventually did instead. Both halves of a training example are already sitting there.

Each looping action becomes a negative example. The first later action in the same rollout that does not loop becomes the positive example paired with it. DeLoop learns from these pairs through direct preference optimization. Because the pairs come from the rollout itself, the method needs no task rewards, no gold answers, and no LLM judges.

The evaluation uses CodeGym with history compression. For Qwen3-8B, DeLoop cuts the share of tasks that contain loops from fifty-seven point three percent to six point one percent. It also uses about one third as many steps per task, with similar task success. At this size, then, the gain is efficiency: the agent reaches comparable outcomes while wasting far fewer steps on calls it has already made.

The larger model gains on success as well. On Qwen3-32B, DeLoop reduces both loops and steps, and raises task success from thirty-seven point seven percent to forty-four point eight percent.

What makes the method cheap is where the signal comes from. A loop announces itself: the same call, the same answer. And the agent's own next move away from it is a ready-made better choice. Pairing the two turns a common failure of long-horizon agents into training data, with nothing external needed to label it. No reward, no reference answer and no grader enter the recipe at any point.