PDD: Parallel Decoding Distillation for Faster Image and Video Generation

Introduction

Diffusion and flow-matching models have become central to modern image and video generation, but their inference process is still expensive. The reason is straightforward: generation usually requires a slow sequence of iterative sampling steps. For images, this increases latency; for video, the cost is amplified by temporal consistency, motion, and the number of frames that must be synthesized.

The paper “Parallel Decoding Distillation for Fast Image and Video Generation” proposes PDD, or Parallel Decoding Distillation, as a way to make existing diffusion and flow models run faster at inference time. Instead of replacing the underlying model family, PDD focuses on distilling a pretrained generator into a form that can produce high-quality outputs with far fewer function evaluations.

Key ideas

• Fewer network evaluations: The paper measures efficiency through NFE, the number of function evaluations during sampling. PDD is designed to work well in the 4-8 NFE regime.
• Multiple denoising steps per call: Conventional samplers progress step by step. PDD accelerates generation by predicting several denoising steps within a single network evaluation, effectively reducing the amount of serial computation.
• A simpler distillation path: Many state-of-the-art acceleration techniques rely on variational score distillation or adversarial losses. While these can deliver high visual quality, they are difficult to optimize and can suffer from mode collapse, reducing video diversity and motion. PDD uses a trajectory-based distillation approach intended to be simpler and more scalable.
• Compatibility with pretrained models: According to the paper, the architecture and training procedure can be applied to any pretrained model and can support sampling with different NFE settings.
• Mean velocity representation: Conceptually, PDD learns a representation of mean velocity without directly regressing its derivative through JVPs or finite-difference approximations.

Why it matters

Fast inference is no longer a secondary concern for generative models. If a video model produces strong results but requires many expensive sampling steps, it becomes harder to use in interactive tools, production pipelines, and large-scale services. PDD addresses this practical constraint by targeting the sampling trajectory itself.

The diversity claim is especially important. Few-step distillation often compresses a rich generative distribution into a narrower set of outputs. In video, this can mean less varied clips and weaker motion. The authors report that PDD improves generated video diversity, suggesting that acceleration does not necessarily have to come at the expense of distribution coverage.

The paper reports state-of-the-art performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. That range of benchmarks indicates that the method is not framed as a one-off optimization for a single architecture, but as a broader distillation strategy for pretrained image and video generators.

Further evaluation and independent reproduction will determine how widely PDD can be adopted. Still, its direction is clear: the next step for generative media may not only be larger models, but faster and more stable ways to use the strong models that already exist.

Source: Hugging Face Daily Papers