Think Less, Locally: Local Preference Learning for Reasoning Compression

Think Less, Locally. Local Preference Learning for Reasoning Compression. By Muyao Yuan and six others, from Xi'an Jiaotong University and WeChat AI, Tencent. Published on the third of November.

Large reasoning models do well on hard tasks, but their long chains of thought are expensive at inference time. Part of the cost comes from a specific habit: they keep deliberating after they already have enough information to answer. Existing methods for compressing reasoning mostly work at the level of whole trajectories. They pay less attention to the local decision, made at a given point, of whether any more reasoning is needed. A trajectory-level view can tell that an answer was long. It is less suited to telling where, along the way, the model already had what it needed and could have stopped.

LoCo, a local preference learning method, targets exactly that decision. Rather than simply encouraging shorter trajectories overall, it teaches the model when to keep reasoning and when to conclude, judged from the state it is in at that moment.

The training data is built from the model's own traces. LoCo detects candidate overthinking states in those traces and freezes the reasoning prefix that leads up to each one. From the same state, it regenerates shorter continuations that are verified. The original and the shorter continuation then form a preference pair conditioned on that state, and the model is trained on these pairs with DPO. Because the prefix is shared, the only difference between the two sides of a pair is what happens after that state, which is exactly the decision the method wants to shape. All of the training data comes from self-generated mathematical traces.

Evaluation covers five mathematical benchmarks and three code generation benchmarks. On mathematics, LoCo cuts completion length by between forty-one point seven and forty-five point four percent. On code, the reduction is between twenty-three point four and thirty-six point three percent. Accuracy is maintained or improved, and LoCo achieves the best AUC on every code benchmark. On mathematics it consistently improves the trade-off between accuracy and efficiency. Code is the notable part, since the model was trained on mathematics alone and the gains still transfer to a domain it never saw during training.

The authors read the transfer as evidence that local preference learning for reasoning compression generalizes across domains. The design choice behind it is to treat overthinking as a decision made at a particular state, not as a property of a whole answer, and to train on exactly those moments where stopping would have been enough.