Why Diffusion Language Models Slow Down Across Tasks and Languages

Diffusion language models are often presented as an efficient alternative to autoregressive generation. Rather than committing to one token at a time, they can update several positions in a single forward pass and decide dynamically how much additional computation is needed. A new study shows why that promise cannot be reduced to a single tokens-per-second number: the cost of decoding changes substantially with the task, the language, and even the eventual correctness of an answer.

What determines the cost?

The researchers analyzed two diffusion models across ten tasks and measured the average number of forward passes used for each generated token. Outputs with a strong connection to the input were generally inexpensive. Copying text or making a light grammatical edit requires relatively little exploration, while open-ended story writing leaves the model with many more possible continuations. In DiffusionGemma, the number of passes per token for story writing was more than three times that required for grammar correction.

Output length alone does not explain the gap. Two properties account for much of the variation. The first is how much of the output is genuinely new compared with the input. The second is the rarity of the words within the task. A continuation that is already strongly determined by context can be settled quickly; a continuation containing unusual or less predictable vocabulary may require more rounds of refinement.

The same pattern appears inside responses that include a reasoning trace. When the final answer largely reuses information already present in the trace, it is relatively cheap to produce. When it introduces a new conclusion or detail, the model tends to spend more computation. This suggests that diffusion decoding is sensitive not only to the prompt, but also to how much uncertainty remains at each stage of a single response.

A multilingual cost penalty

The study also finds that language affects more than the number of output tokens. Languages that are split into more tokens require additional forward passes per token as well. They therefore face a compounded cost: the same content may occupy more tokens, and each token may take more decoding effort. This is an important consideration when comparing the practical efficiency of diffusion models across languages, since an English-centric benchmark may understate the cost of other language settings.

Speed as a quality signal

Perhaps the most practical result concerns answer selection. On the same mathematics problem, incorrect answers tended to require more computation per token than correct answers. The authors used this relationship to weight candidate answers by decoding speed instead of treating every sample equally. This simple strategy beat majority voting by up to 4.2 percentage points without requiring extra inference.

The finding does not mean that a fast answer is always correct. Speed is a signal, not a proof. The experiments cover particular models and tasks, so the relationship may vary across domains and decoding configurations. Still, the work points toward a broader design principle: inference systems can use the model’s own computational effort to allocate resources, rank candidates, or detect uncertain outputs.

Why it matters

Diffusion models may have their strongest efficiency advantage when the desired output is already constrained by the input or by an existing reasoning trace. Creative generation, rare vocabulary, and multilingual tokenization can reduce that advantage. Understanding these differences is essential for realistic serving-cost estimates and fair cross-language comparisons.

More broadly, the study reframes decoding effort as observable information. Instead of treating computation as a hidden expense, future systems could use it to guide candidate selection and adaptive inference. Whether this signal transfers reliably to larger models and wider tasks remains open, but it offers a promising route toward cheaper and more dependable generation.

Source: Hugging Face Daily Papers