vLLM Pushes Speculative Decoding Beyond One-Token-at-a-Time Drafting

Introduction

Speculative decoding has become one of the practical ways to speed up large language model serving. The basic idea is straightforward: a draft model proposes multiple candidate tokens, and the main verifier model checks them in a single forward pass. This can reduce the impact of memory-bandwidth bottlenecks and improve serving throughput. The latest vLLM Blog post focuses on a next step in that direction: open-source support in Speculators and vLLM for P-EAGLE, DFlash, and DSpark, three algorithms built around parallel drafting.

Key points

• The old bottleneck sits inside drafting itself. Methods such as EAGLE and MTP improved speculative decoding by allowing the draft model to use the verifier model’s hidden states. That made draft proposals more accurate. But even advanced EAGLE-style systems still rely on autoregressive drafting: producing several draft tokens requires generating them one after another.
• Parallel drafting changes the cost structure. P-EAGLE, DFlash, and DSpark aim to predict a full candidate block in one draft forward pass. This reduces the dependence between draft latency and speculation length, making it easier to use more expressive draft models and lessening the operational need to constantly tune the lookahead length.
• P-EAGLE extends the EAGLE line. It directly consumes verifier hidden states as features and maps them to multiple future positions at once. To make training manageable, it uses draft block sparsification, reducing loss computation along the lookahead dimension and focusing more on near-term future tokens.
• DFlash injects verifier information through the KV-cache. Instead of appending hidden states to the input sequence, DFlash projects them into the speculator model’s KV-cache. This lets the attention mechanism condition tightly on verifier state without expanding input length. For training, it uses sequence length sparsification by computing block prediction losses only at sampled anchor points.
• DSpark adds correction and filtering. Built on a DFlash-style backbone, DSpark introduces a lightweight autoregressive correction head to improve sequential consistency. It also adds a confidence head that estimates which draft tokens are likely to be accepted, so fewer low-value candidates need to be sent to the verifier.

Why it matters

The importance of parallel drafting is not only that it may be faster. It changes the engineering trade-off behind speculative decoding. If the draft model runs once per block rather than once per token, teams can consider larger and more capable speculators without paying the same sequential latency penalty. It may also make serving systems easier to operate because speculation length becomes less tightly coupled to draft cost.

The blog does not claim a universal speedup across every workload. It shows examples involving Qwen3-8B with P-EAGLE on math reasoning, Qwen3-30B-A3B with DFlash on coding, and gemma-4-31B-it with DSpark on coding, and reports meaningful gains compared with EAGLE-3. But it also notes that results depend on model, task, and hardware configuration.

For teams working on LLM serving, the update is notable because it reframes speculative decoding from “how can a small model guess better?” to “how can candidate generation stop being serial?” If these methods generalize well in real deployments, parallel drafting could become a common component in high-throughput inference stacks.

Source: vLLM Blog