Revisiting Complete Reasoning Traces for Post-Training
Jaehui Hwang, Sangdoo Yun, Byeongho Heo, Dongyoon Han
NAVER AI Lab
Published September 7, 2026 · on the feed September 10, 2026 · ▲ 22 · arXiv 2609.07103
The abstract
Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.
The Brief
2:17A short podcast summary of the paper. Written and voiced by AI.
Transcript
Revisiting Complete Reasoning Traces for Post-Training. By Jaehui Hwang and three others, from NAVER A I Lab. Published on the seventh of September. Its code repository is a placeholder: a readme, and nothing to run yet.
Language models are often post-trained on reasoning trajectories collected in advance, to improve how they reason. Those trajectories tend to be long, because the paths through a problem are complex and interwoven, often taking detours on the way to the answer. Whether a model actually benefits from learning each trajectory in full, during supervised fine-tuning for instance, has been little examined. The claim here is that it mostly does not. Complete trajectories give only limited benefit, while partial ones remain effective even when heavily truncated.
A pilot study came first, comparing full trajectories against partial ones, and it is there that the limited benefit of complete traces first showed up. The trajectories were then probed for redundancy in two ways. Attention-based analyses looked at what the model draws on. Controlled studies removed tokens and measured the effect. Both pointed the same way: the intermediate tokens contribute minimally to the quality of the final reasoning.
The authors offer an interpretation. Leaving out redundant information may let a model infer coherent alternatives internally, filling in the missing steps from its own knowledge, provided it knows where the trajectory starts and where it ends. In line with that, training on endpoints produces consistent changes in reasoning behaviour. It also helps post-training methods built on reinforcement learning or on on-policy distillation.
The central evidence concerns what intermediate tokens contribute, measured through attention and through removal. The explanation that models reconstruct the missing steps is phrased as a suggestion, something the results make plausible rather than something shown directly. What is shown is the pattern: limited gain from full traces, robustness to truncation, and benefit from endpoint training carrying over to reinforcement learning and distillation.
Revisiting is exactly what the authors recommend. The long, winding traces that post-training datasets are built from may be carrying less of the value than assumed, and the authors call for revisiting complete reasoning traces as the default. If the middle of a trace can be cut without much loss, then what a model learns from it lies mostly in knowing the start and the destination. Code is available.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Long traces are the standard diet for reasoning models. Is all of it nourishing?
You are post-training a language model on reasoning traces collected in advance. Each trace runs from a question to its answer, and many are very long, wandering and backtracking on the way.
The road ahead
- 1 · The problem ~6 min Models are post-trained on long, wandering reasoning traces, and it is underexplored whether every token of that wandering actually helps.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min The middle of a reasoning trace is largely redundant; the beginning and ending carry what matters, so training can keep just those ends.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min A pilot study, two redundancy probes, endpoint training for SFT, then the same masking carried into reinforcement learning and distillation.
- 6 · The experiment ~6 min You’ll guess: When fine-tuning a model on collected reasoning traces, how did complete trajectories compare with heavily cut partial ones?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.