WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models
Hyeongju Ha, Jae-Joon Kim
Seoul National University
Published September 26, 2026 · on the feed September 29, 2026 · ▲ 7 · arXiv 2609.23033
The abstract
Looped language models repeatedly apply a weight-shared block to increase effective depth without increasing parameter count, but the resulting T sequential recurrent-block calls per generated token substantially increase decoding latency. To address the issue, we introduce Wavefront Decoding (WFD), a training-free self-speculative decoding framework designed for looped language models. WFD exploits two properties of these architectures: intermediate recurrence outputs provide effective draft predictions, and weight sharing allows token states at different positions and recurrence depths to be processed in one batched recurrent-block call. WFD organizes these mixed-depth states into a diagonal wavefront, continuously drafting new positions at shallow depth while advancing earlier positions toward full-depth verification. Unlike the phase-separated draft-then-verify schedule, WFD therefore concurrently batches drafting and verification within the same recurrent calls, while rejected drafts are corrected using full-depth predictions. Across six Spec-Bench task categories, WFD achieves 2.42x speedup on Ouro-2.6B and 3.54x on Huginn-3.5B over autoregressive decoding, consistently outperforming draft-then-verify. Cross-recurrence KV sharing further reduces wavefront KV traffic and increases WFD's speedup to 4.81x on Huginn-3.5B. The code is available at https://github.com/summerbro-hhj/wavefront-decoding.
The Brief
2:17A short podcast summary of the paper. Written and voiced by AI.
Transcript
WaveFront Decoding. Parallelized Self-Speculative Decoding for Looped Language Models. By Hyeongju Ha and one other, from Seoul National University. Published on the twenty-sixth of September. Its code repository is worked out: about fifty source files, with a readme.
Looped language models get more effective depth without more parameters by applying the same weight-shared block over and over. The bill arrives at decoding time. If the block is applied T times, every generated token needs T recurrent-block calls in sequence, and that substantially increases latency. The saving in parameters is paid back, token by token, in waiting, and the deeper the loop runs, the longer each token takes to arrive.
Wavefront Decoding, or WFD, is a training-free self-speculative decoding framework built for these models. It leans on two properties of looped architectures. First, the outputs of intermediate recurrences already make effective draft predictions. Second, because the weights are shared, token states at different positions and different recurrence depths can all be processed together in one batched call to the recurrent block. The model is its own drafter, and no training is needed to make it one.
Put those together and you get the wavefront. WFD arranges the mixed-depth states along a diagonal. New positions are continuously drafted at shallow depth, while earlier positions keep advancing toward full-depth verification. Picture a diagonal front sweeping forward, shallow at its leading edge and deep at its trailing edge. A conventional draft-then-verify schedule runs those as separate phases. Here drafting and verification are batched concurrently inside the same recurrent calls, and any rejected draft is corrected using the full-depth prediction.
Across six Spec-Bench task categories, WFD speeds decoding up by two point four two times on Ouro-2.6B and three point five four times on Huginn-3.5B, measured against autoregressive decoding, and it consistently outperforms draft-then-verify. Adding cross-recurrence KV sharing cuts the key-value traffic of the wavefront and raises the speedup on Huginn-3.5B to four point eight one times.
The weight sharing that makes a looped model slow to decode is also what WFD exploits. Since every depth runs the same block, a shallow draft and a deep check can ride in the same batch, and the loop stops being a queue.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is a model that thinks deep but answers slowly.
You run a looped language model: one weight-shared block applied T times to gain depth without adding parameters. The bill comes at decoding: each token needs T block calls in a row.
The road ahead
- 1 · The problem ~7 min A looped model saves parameters by reusing one block, but every token then waits for that block to run T times in a row.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Let shallow passes guess the next tokens while deeper passes check earlier ones, all inside the same batched block calls.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Draft from shallow outputs, batch mixed-depth states on a diagonal, deepen earlier ones to full depth, and correct any rejected draft.
- 6 · The experiment ~6 min You’ll guess: With no training, how much faster than plain autoregressive decoding was Wavefront Decoding on the looped model Ouro-2.6B?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.