Decoding Looped Transformers Better for (Almost) Free
Weihao Liu, Huangjie Zheng, Tianrong Chen, Rohit Dilip, Richard He Bai, Yizhu Jiao, Yuyang Wang, Ruixiang Zhang
Apple
Published October 1, 2026 · on the feed October 2, 2026 · ▲ 43 · arXiv 2610.02185
The abstract
Looped Transformers achieve parameter efficiency by repeatedly executing a shared block across recurrent loops. Each loop yields an intermediate representation decodable for the same next token, yet standard decoding discards earlier states. Because earlier loops embody less computation, recurrence inherently supplies aligned weak-and-strong prediction pairs without auxiliary models or external training. We introduce LoopCD, a training-free contrastive decoding framework that guides token selection by contrasting the final prediction with an earlier recurrent pass, operating either in logit space with one extra output pass (LoopCD-Logits) or in hidden-state space with zero output overhead (LoopCD-Hidden). Across four looped Transformer families, LoopCD delivers substantial, consistent gains at full recurrent depth: LoopCD-Logits raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%, while LoopCD-Hidden lifts Huginn's HumanEval pass@1 from 22.56% to 31.71%. Crucially, these performance gains enable halving the number of recurrent loops while still matching or exceeding full-depth unguided baselines, reducing forward FLOPs by 22.5% to 48.2%. By transforming intermediate recurrent states into effective guidance signals, LoopCD achieves superior decoding quality while substantially reducing inference compute.
The Brief
2:08A short podcast summary of the paper. Written and voiced by AI.
Transcript
Decoding Looped Transformers Better for (Almost) Free. By Weihao Liu and seven others, from Apple. Published on the first of October. No code repository is linked to it.
Looped Transformers save parameters by running one shared block again and again across recurrent loops. Every loop produces an intermediate representation that could be decoded into the same next token. Standard decoding uses only the last one and throws the earlier states away. The model has, in effect, already drafted several answers to the same question, and all but the final draft are discarded unread.
Those earlier states are not useless. An earlier loop has done less computation, so the recurrence naturally supplies pairs of predictions, one weaker and one stronger, aligned on the same token, with no auxiliary model and no extra training. LoopCD is a training-free contrastive decoding framework built on exactly that. It guides token selection by contrasting the final prediction against an earlier recurrent pass.
There are two versions. LoopCD-Logits does the contrast in logit space, at the cost of one extra output pass. LoopCD-Hidden does it in hidden-state space, with zero output overhead. Both draw their weak signal from inside the same model, during the same forward computation, so neither needs a second network to be trained or loaded.
The method was tested across four looped Transformer families. At full recurrent depth, LoopCD-Logits raises the AIME two thousand and twenty-four pass at one of Ouro-2.6B-Thinking from sixty-one point eight eight percent to seventy-three point three three percent.
LoopCD-Hidden lifts the HumanEval pass at one of Huginn from twenty-two point five six percent to thirty-one point seven one percent. The gains are large enough to spend on speed. Halving the number of recurrent loops still matches or beats the full-depth unguided baselines, which cuts forward FLOPs by between twenty-two point five and forty-eight point two percent.
Ordinary decoding treats the early loops as scaffolding to be discarded. LoopCD reads them as a built-in weaker opinion, and uses the distance between weak and strong to steer. Better answers and less compute come out of the same move, which is what the almost free in the title is pointing at.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model drafts several answers and throws all but one away. Start there.
You serve a looped language model: it runs one shared block again and again before emitting each token. Every loop produces a state that could be decoded into the same next token.
The road ahead
- 1 · The problem ~7 min A looped model drafts the next token once per loop, but standard decoding reads only the last draft and wastes the rest.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min LoopCD treats an earlier loop as a free weak guess and pushes the final guess further away from it.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Run the loops, keep an early pass, contrast the final prediction with it, then pick the token from the guided result.
- 6 · The experiment ~6 min You’ll guess: Running only half the recurrent loops, how did LoopCD compare with the full-depth unguided baselines?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.