SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

SAGE. Mitigating Long-Horizon Reasoning Biases via Topological Guidance. By Xinyue Zeng and three others, from Virginia Tech, the University of Wisconsin Madison, and one other institution. Published on the twenty-fourth of September. Its code repository is worked out: about forty source files, with a readme.

Long-horizon reasoning remains a central challenge for large language models, especially when rewards are sparse. The authors argue that the brittleness comes from two biases, both induced by the structure of complex reasoning spaces. The first is an exploration bias. Models are drawn toward branches that look plausible locally but are structurally unstable. The second is a compounding bias. Small local deviations accumulate as reasoning goes deeper, and they end up suppressing rewards that were rare to begin with.

To study this, they introduce Symbolic Closure Analysis, or SCA. It serves two purposes. It is a theoretical lens, characterizing how branching structures and sparse rewards produce these two biases in long-horizon reasoning with local admissibility. And it is a design principle for building structural priors into reasoning tasks that are less formal. The first role explains why long-horizon reasoning breaks. The second turns that explanation into something a method can be built from, which is exactly how the paper uses it.

Guided by that analysis, the paper proposes SAGE, short for Structural Admissibility-Guided Exploration. It is a unified framework that injects structural guidance into long-horizon reasoning, with one kind of guidance aimed at each bias.

Against exploration bias, SAGE uses algebraic sparsification. Candidates that are locally admissible are projected onto operator-indexed algebraic subspaces, which suppresses spurious branching. Against compounding bias, it uses hyperbolic structural guidance. Reasoning states are embedded into a negatively curved space, and that embedding provides dense signals along the depth of the reasoning, where otherwise the reward would be sparse. The two mechanisms are designed to complement each other: one prunes the width of the search, the other supports its depth.

Evaluation covers twelve benchmarks and seven model families, and SAGE outperforms competitive baselines across them. The largest figure comes from the Andrews-Curtis problem, which the authors describe as an open, real-world, long-horizon task. There, SAGE achieves up to an eight-fold improvement, on a problem the authors call open. And the spread of the rest, across seven families of model, means the gains are not a property of one particular model.

Two biases were named, and each gets its own remedy: sparsification for the pull toward unstable branches, and a curved embedding for the errors that pile up with depth. Both are structural, which is the thread running from the paper's analysis to its method.