SWE-Duel: Towards a Self-Scaling Arena for Coding Agents

SWE-Duel. Towards a Self-Scaling Arena for Coding Agents. By Wei Minn and three others, from Singapore Management University. Published on the eighth of October. Its code repository is worked out: about one hundred and thirty source files, with a readme and a licence.

A static coding benchmark is fixed on the day it is released. As time passes, its tasks leak into training data, and stronger agents eventually saturate it. The authors argue for a different target: a benchmark that is self-scaling. Its tasks and its oracles are created at evaluation time, so there is no content sitting around in advance that could leak. And its difficulty is set by the agents taking part, not frozen at release. Both failures of a static benchmark come from the same source, which is that its contents exist before the evaluation does. A self-scaling benchmark removes that source.

SWE-Duel is an arena where two coding agents evaluate each other, in mirrored turns with a Red side and a Blue side. Red authors a task with a hidden defect. Blue tries to find and fix it without being told it is there. Then the agents swap roles, so each is measured both as a task author and as a solver.

In each turn, the Red agent works on a real repository across three context-isolated sessions. It proposes and implements a new feature, with feature tests. It then embeds a subtle bug in that feature, along with sealed tests that trigger the bug. The resulting pull request is admitted only under two conditions. It must pass the feature tests and the repository's existing tests, while failing exactly the bug tests. And a fresh session of the same agent must be able to repair the bug blind, which rules out unsolvable tasks. That second gate matters for fairness. Red cannot win simply by planting something no one could fix, because its own fresh session has to fix it first.

The Blue agent then reviews the pull request, with no word that a bug exists. It wins only if its revision passes all three test suites. The tournament behind the results has eleven model-harness participants, working on fifteen repositories in five languages, over one thousand five hundred turns.

The leaders win in different ways. GPT-5.5 with Codex CLI ranks first, as the strongest solver: it repairs ninety-eight point three percent of the defects it reviews. Claude Opus 4.8 with mini-swe-agent ranks second, as the strongest task author, with a Red win rate of twenty-seven point three percent.

One more comparison stays inside a single model family. Tasks written by the newest version defeat a fixed-reference solver far more often than tasks written by older versions: forty-two point nine percent of the time, against four percent. The authors present this as preliminary evidence that task difficulty scales with the ability to author tasks.

That last result is the self-scaling claim in miniature. If a better model writes harder tasks, then the arena's ceiling rises with its participants, and the benchmark does not need to be rebuilt each time agents catch up. Solving and authoring also come apart in the ranking: the agent that repairs the most is not the one whose tasks are hardest to beat.