Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

Root-Cause Attribution Is a Search Problem. Continual Search for Long-Horizon Agent Failures. By Harsh Raj and nine others, from Scale AI. Published on the eleventh of September. No code repository is linked to it.

AI agents deployed on long-horizon tasks produce massive execution logs. Diagnosing failures within those records is crucial for reliability, because a diagnosis turns an outcome-level signal, the fact that something failed, into interventions someone can act on. The sheer scale of the data makes human review impractical, which drives the need for automated root-cause attribution, or RCA.

Automated RCA with LLMs, however, has low diagnostic accuracy, especially as execution traces grow larger. The relevant information is often sparse, distributed across distant actions, and disconnected from the visible failure. That reduces root-cause attribution to a massive search problem. Existing methods typically rely on a one-shot LLM judgment over the trace. For shorter trajectories that works. On longer ones, the judge tends to settle early on a plausible diagnosis, and critical evidence later in the trace goes unexamined. A judge that commits early is, in effect, searching only the part of the trace it happened to weigh before deciding.

Continual Search is an iterative framework built around that failure. Over successive turns, it nudges the judge to keep searching for diagnostic evidence that is still unresolved. The diagnosis is no longer one verdict delivered in a single pass, but a search carried on across turns. Each further turn is a push to find evidence that the diagnosis so far has not accounted for.

The framework is evaluated on four existing RCA benchmarks. The authors also recognize that current benchmarks lack massive execution traces, so they introduce MegaRCA-Mix to evaluate attribution at scale. It is a challenging testbed of fifty human-annotated failure trials, spanning long-horizon tasks that are heavy on execution: exactly the setting where a single early judgment is most likely to miss something. The existing benchmarks show whether the method helps in general. MegaRCA-Mix asks whether it helps where traces are longest, which is where one-shot judges were said to struggle.

Across multiple benchmark suites and model families, Continual Search consistently improves attribution performance. On MegaRCA-Mix, for example, it improves GPT-5.5's F1 score by more than forty percent, from zero point three four nine to zero point four nine eight. In rounder terms, F1 moves from about a third to about a half.

The authors single out one result as more interesting still. Within the same model family, lower-tier models can even surpass their higher-tier counterparts. In their reading, effective search supersedes raw model scale.

The title makes the argument in one line: root-cause attribution is a search problem. If the evidence for a failure is scattered and far from where the failure shows, then how thoroughly a long trace gets searched can matter more than which model does the searching, and the tier reversal is the authors' evidence that it does. The fix asks nothing new of the model itself. It changes how long, and how insistently, the model is made to look.