Diffusion Language Models Change Speed Across Tasks and Languages

Diffusion Language Models Change Speed Across Tasks and Languages. By Mohsen Fayyaz and four others, from UCLA, LMU Munich, and one other institution. Published on the nineteenth of October. No code repository is linked to it.

Diffusion language models can write several tokens in one forward pass, and they decide during generation how many passes to spend. That means two outputs of the same token length can vary a lot in what they cost. The claim is that this variation follows a pattern. The number of forward passes per token depends heavily on the task. Text copied or lightly edited from the input is cheap. Open-ended writing is costly. Put more generally, how fast a diffusion model decodes depends on how much of the output is already determined.

The authors analyze two diffusion models across ten tasks. In DiffusionGemma, generating a token for story writing requires more than three times as many forward passes as generating one for grammar correction. Two properties of the output explain much of that gap. The first is how much of the output is new relative to the input. The second is how rare its words are within the task. Both fit the cheap and costly ends of the spectrum. Grammar correction mostly reproduces the input with familiar words, so little of it is new. A story is almost entirely new, and its vocabulary is far less predictable.

The same pattern shows up inside a single response when thinking is involved. The final answer is cheap when it repeats the model's reasoning trace, and costly when it adds something new. Multilingual generation shows a related effect. Highly tokenized languages also require more passes per token, which stacks on top of the cost those languages already pay by needing more tokens in the first place. Those languages are charged twice: once in the number of tokens, and again in the passes each token takes.

The last result connects cost to correctness. On the same math problem, wrong answers take more passes per token than correct ones. That difference can be put to work. Weighting candidate answers by their decoding speed beats majority voting by up to four point two points, and it does so at no extra cost, since the speed is observed during decoding anyway.

The evidence rests on two diffusion models and ten tasks, and the only model the abstract names is DiffusionGemma, where the three-times story-writing figure was measured. The two output properties are said to explain much of the cost gap, not all of it. The link between speed and correctness is stated for answers to the same math problem, and the gain over majority voting is an upper figure: up to four point two points.

The broader reading the authors offer has two sides. Diffusion's computational advantage is conditional: it is largest when much of the output is already pinned down by the input or by earlier reasoning, and shrinks for genuinely new text. Meanwhile, the decoding effort a model spends is itself informative, a practical signal for making inference both more efficient and more reliable.

Diffusion Language Models Change Speed Across Tasks and Languages. By Mohsen Fayyaz and four others, from UCLA, LMU Munich, and one other institution. Published on the nineteenth of October. This is the long version: the whole paper rather than the abstract.

An autoregressive language model pays one forward pass for every token it writes, so its cost is simply the length of its output. A diffusion language model works differently. It refines a block of positions over several steps and commits a variable number of tokens at each one, and it decides during generation how many steps to spend. Two outputs of exactly the same length can therefore cost very different amounts. That speed is usually reported on math and code alone. The question here is where a diffusion model actually spends its forward passes, and why.

Two models are studied. DiffusionGemma is a mixture-of-experts model trained with uniform diffusion, writing blocks of two hundred and fifty-six tokens. DreamReasoner is a smaller dense model based on masked diffusion, writing blocks of thirty-two. Both decode block by block, and both commit only the tokens that meet a confidence target, which is what makes their step counts adaptive. The cost measure is forward passes per hundred output tokens. An autoregressive model always scores a hundred. A diffusion model that writes twenty tokens per step scores five. The tasks are chosen to constrain the answer in different ways. Reproducing tasks, grammar correction and extractive summarization, mostly copy the input. Chained tasks, code editing, code generation and competition math, build later steps on earlier ones. Open-ended tasks, real user queries from WildBench and story writing, leave the output largely undetermined. Three rewriting tasks are added later: German to English translation, clinical notes from doctor and patient conversations, and abstractive chapter summaries.

The core is an attempt to explain the gap with two properties of the output, measured on words rather than model tokens so they do not depend on any tokenizer. The first is novelty. The output is scanned left to right and split greedily into the longest fragments copied from the input, with every uncopied word counted as its own segment. Novelty is the number of segments per output word. A ten-word output made of two copied fragments and three new words has five segments, so its novelty is one half. The second property is surprisal: how rare the output's words are within the task. The authors pool each model's outputs for a task, count word frequencies with add-one smoothing, and average the information content of each word in bits. Code reuses keywords and boilerplate, so even novel code is unsurprising. Then the two are multiplied into a single score, new information per word.

That product is what tracks cost. For DiffusionGemma, novelty alone correlates with cost at plus zero point seven two across the ten tasks, and surprisal alone barely at all. The product reaches plus zero point seven six. For DreamReasoner the same pattern runs from plus zero point seven nine to plus zero point eight five. The clearest illustration is math and code against abstractive summaries: all three are mostly new text, but math and code use far less surprising words and cost about six passes per hundred tokens against nine and a half. The fit has honest misses. Translation has the new information of a story yet costs much less, because it copies content across languages, which word matching cannot see. Stories and summaries cost more than predicted, because their unpredictability lives at the level of discourse.

The task hierarchy is clean. In DiffusionGemma, a story token costs three point one times a grammar correction token: eleven point four three passes against three point seven two. DreamReasoner pays much more overall, because of its smaller blocks, yet ranks the task groups the same way. With thinking switched on, the same logic appears inside a single response. The final answer can copy from the model's own reasoning, and the less novel it is relative to that reasoning, the cheaper it is compared with it, at a correlation of plus zero point eight nine across tasks. In grammar correction the answer is far cheaper than the thinking. On stories it costs more, because the sentences are still written for the first time.

The comparison that most directly tests the idea that cost reflects remaining uncertainty is on math. Correct final answers stay equally cheap from easy to hard problems, about four passes per hundred tokens. Wrong ones climb from about five to about six. Holding the problem and output length fixed across random seeds, wrong runs still need about point eight more passes. Weighting each sampled answer's vote by how fast it was decoded beats majority voting by four point two points with two samples and two point three with five, from seventy-one point four to seventy-three point seven percent. Languages show a double tax: Armenian needs more tokens than Spanish and also more passes per token, and across twenty languages the two rise together.

The paper has a short limitations section with one point: only two models, because adaptive diffusion decoders strong enough for reasoning and long-form writing are very new. DreamReasoner appeared in June of this year, and DiffusionGemma two months later. The evidence has further edges the authors flag themselves. The correlations rest on ten tasks, with uncorrected p-values described as indicative only. Problem position within a contest is a coarse stand-in for difficulty. Each task item was run once, with one seed. Exclusions matter too: many DreamReasoner stories hit the token limit in repetition loops, and keeping them would have made its stories look cheaper than its code. The multilingual study uses DiffusionGemma alone.

The idea to carry away is that a diffusion model's speed is not a property of the model alone. It depends on how much of the output is already determined: by the input, by the task's habits, or by the model's own earlier thinking. And because effort follows uncertainty, the pass count the decoder keeps anyway doubles as a measure of confidence, one that already improves answer selection at no extra cost.