On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction

On the SPOT. Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction. By Eliezer Shlomi and three others, from the Technion, Israel Institute of Technology and Ben-Gurion University of the Negev. Published on the twenty-sixth of October.

Diffusion language models, or DLMs, hold out the promise of generating text in parallel. In practice they face a speed and quality trade-off that has not been resolved. Exact decoding fills in only one masked position per model pass, which is slow. Faster schemes that decode several tokens at once are lossy. For autoregressive models, the standard way to accelerate without loss is speculative decoding. It does not carry over directly, though, because a DLM has to choose two things at each step: what to generate, and where.

The authors argue that the order is the missing piece. Their observation is that the model's own predictions from the previous pass already give the right token at over ninety percent of positions. What those predictions do not give is which position comes next. The content is largely known in advance; the schedule is not. So the thing to speculate about is the order itself, the sequence in which masked positions get committed. Speculating Positions Order Trajectory, or SPOT, is a small model built for exactly that job, with zero point six two million parameters.

SPOT reads only the target model's output distributions. From them it predicts the commit order and proposes a tree of candidate continuations. The target then verifies that tree in a single pass, by checking state equality. To keep the verification exact at any temperature, the authors use what they call coupled-noise match: the drafter and the target share the same Gumbel noise. That matters because sampling at a higher temperature is random, and an exact check needs both sides drawing from the same randomness.

The same thinking leads to a second variant, SPOT-Sets, for dynamic threshold decoding. In that regime a single pass commits several positions at once, so SPOT-Sets drafts whole commit sets together with their tokens rather than one position at a time.

On SDAR-8B, SPOT cuts the number of target passes by up to six point eight times, and reaches up to three point four times the throughput of standard decoding. Fewer target passes is the direct saving, since every pass is a full run of the target model. SPOT matches or beats both calibrated and training-free drafting baselines in every task and every budget tested, at temperature zero and at temperature one.

The learned drafter also transfers across models. The authors show successful transfer to six different DLMs, and with a light calibration the SPOT-Sets drafter transfers as well. Layered on top of dynamic threshold decoding, SPOT-Sets improves the throughput of SDAR-8B by fifty-nine percent on average over the threshold decoder alone.

Speculative decoding needed a new question before it could work for diffusion models. For an autoregressive model, the position is fixed and only the token is in doubt. Here the authors found the token mostly settled already and the position open, and built a deliberately tiny drafter to guess it. Because the target still checks every proposal, and the shared noise keeps that check exact, the speedup comes without giving up the exact output. The parallel text generation that diffusion models promised, held back until now by a choice between slow exact decoding and fast lossy decoding, gets a third option: fast and exact, with the drafter carried from one model to the next.