StopLoss: Teaching Agents to Predict Failure and Stop Early

StopLoss. Teaching Agents to Predict Failure and Stop Early. By Jiarong Liang and five others, from the University of Waterloo. Published on the eighteenth of October. Its code repository is public and real: about thirty source files.

Coding agents keep working on runs that are going to fail verification anyway. Every further step on such a run spends compute on an outcome that is already lost. The obvious fix is to have a frontier model watch the run and call it off, but that watcher is expensive in its own right, and its cost is paid on every check, including all the checks on runs that would have succeeded.

StopLoss replaces that expensive watcher with a small external monitor that learns to recognise failure from completed trajectories and the outcomes a verifier gave them. The name comes from the stop-loss order in finance. A trader sets one to get out of a position before the losses grow, and StopLoss ends a run early when the run is likely to fail, which saves resources.

The training signal comes from final outcomes. Whether a whole run passed or failed is used to label earlier local windows within it. That lets a fine-tuned language model score the most recent tool steps in a single forward pass. To turn the score into a decision, a threshold is calibrated on successful runs, and crossing it triggers an abort. None of this changes the executor. The agent doing the coding is left as it was, and the monitor sits outside it.

The monitor is trained on GPT-family trajectories from SWE-rebench, then evaluated on SWE-bench Verified and on a subset of SWE-bench Pro, with both GPT and Claude executors. It is compared against six general-purpose judges that read identical windows.

Outcome supervision lifts failure prediction from near chance to a useful signal at the very first check, and each check costs one to two orders of magnitude less than with any tested judge.

Those cheaper checks turn into real savings. Measured against simply completing every task, StopLoss lowers the cost per retained solve by four point five percent on Verified, while preserving ninety-nine percent of successful runs. On the costlier Pro subset the saving is up to thirty-one percent, and with a Qwen3-8B backbone it reaches forty-eight percent. In all four settings, its cost per retained solve is below that of every judge policy.

Early stopping only helps if spotting failure costs less than the work it cuts short. A large judge can see failure coming, but each look is too expensive. StopLoss is trained only on how past runs ended, and that makes each check cheap enough for stopping early to pay for itself. The verdicts of finished runs become the lesson the small monitor learns from.