Contextual Correction of Parallel Drafts for Speculative Decoding

Contextual Correction of Parallel Draftsfor Speculative Decoding. By Roy Eisenstadt and seven others, from a stealth startup in Tel Aviv, Tel Aviv University, and one other institution. Published on the eighth of October. Its code repository is public and real: twelve source files.

Speculative decoding speeds up inference in large language models. A lightweight drafter proposes several tokens, and the target model then verifies them in parallel. Recent drafters based on diffusion make this especially attractive. Their deep, attention-based backbones predict well, and because they draft several future positions in parallel, they avoid the sequential cost of autoregressive generation. That parallelism has a price. Within a draft block, each token is predicted without knowing which tokens were actually chosen at the positions before it. The dependencies inside the block, which autoregressive generation would supply for free, are gone. Each position makes its best guess alone, and nothing checks whether the guesses fit together as a sequence.

DCOCO, the DFlash Contextual Corrector, is a lightweight mechanism meant to put those dependencies back without giving up parallel drafting. The choice it changes is a small one. Rather than committing to the top prediction at every position, it keeps several candidate tokens alive. A single top choice per position leaves nothing to correct. Several candidates per position leave room to pick the ones that belong together.

Those candidates are then checked against each other. DCOCO evaluates how compatible they are in context across neighbouring draft positions, and it does this evaluation in parallel too. Token selection becomes informed by context, and the extra cost stays low. The point of doing the check in parallel is that the drafter's main advantage survives. The dependencies come back, but the sequential cost that autoregressive drafting would carry does not.

The evaluation spans four verifier backbones and ten benchmarks. In every configuration tested, DCOCO has the highest average throughput. That covers greedy and stochastic decoding, with extended reasoning and without it. No setting in that grid has a different winner. At its best, the speedup reaches six point six times over vanilla decoding.

The advantage holds under concurrent serving. With between four and thirty-two concurrent requests, DCOCO beats the strongest competing method by six to twenty-three percent. At those same loads, other correctors that work sequentially fall below DFlash with no correction at all.

The result to hold on to is the serving one. A corrector that runs sequentially can undo its own gains once the system is busy, while one that scores context in parallel keeps paying off. Restoring the dependencies inside a block, in parallel, gave steady acceleration across decoding regimes and across serving loads.