SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models

SpecFold. Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models. By Chung-En Ho and five others, from the Georgia Institute of Technology. Published on the sixth of October. Its code repository is public and real: about thirty-five source files.

Diffusion large language models generate text by denoising it block by block, over several iterations. Multi-branch speculative decoding speeds this up by verifying a main branch together with several draft branches in a single forward pass. Earlier acceleration methods for these models mostly exploit temporal redundancy, the repetition across denoising steps. The authors point to a second kind that sits inside each verification step. Draft branches inherit most of their tokens from a parent and unmask only a few extra positions, so large portions of the hidden states stay highly similar from one branch to the next. Verification computes them again anyway.

SpecFold is an algorithm and system co-design that targets this multi-branch computational redundancy. The aim is to cut the cost of verifying many branches by reusing what the branches already share, rather than recomputing it once per branch.

On the algorithm side, SpecFold performs token-level residual gating and selectively reuses parent computation through folded attention and FFN, while preserving the residual hidden states. On the system side, a Triton kernel implementation turns this fine-grained reuse into real end-to-end throughput, through efficient sparse execution across branches. The method is orthogonal to temporal caching, and compatible with existing speculation strategies for diffusion language models, so it can be stacked with what is already in use.

The evaluation covers two families of diffusion language models, five models in all, and five standard benchmarks. SpecFold reaches up to one point six four times the throughput of Spiffy. Against vanilla decoding the gain is up to one point nine nine times. In both cases task performance stays comparable, so the speed is not bought by giving up accuracy. The spread of the evaluation matters too. Holding up across two model families rather than one suggests the redundancy SpecFold exploits is a property of multi-branch verification in general, not a quirk of a single architecture, which is consistent with the authors' description of it as a complementary redundancy axis.

The contribution is a new place to look for savings. Prior work squeezed the time axis, step to step. SpecFold looks sideways, across the branches checked within a single step, finds that they are mostly copies of one another, and skips recomputing what they share. Because the two axes are separate, the savings can add up.