PixVid Brings Video Diffusion Back from Latent Space to Pixels

Introduction

Most modern video generation systems do not operate on raw pixels. They first use a VAE to compress video across both space and time, then run diffusion in the resulting latent representation. This makes training and inference more manageable, but the compression introduces important trade-offs. Increasing resolution quickly expands the token budget, control signals are often applied to groups of frames rather than individual frames, and an interactive system may need to wait for an entire multi-frame block before returning a response.

PixVid, presented by a Roblox research team, explores a different direction: lifting an existing latent video model into pixel space. Rather than rebuilding the generator from scratch, the method preserves the main diffusion Transformer and changes the output path that turns its representations into video.

What changes in PixVid?

• The diffusion backbone is retained. PixVid is fine-tuned from Wan2.1-14B, allowing the original Transformer to continue handling spatiotemporal modeling and denoising.
• The VAE decoder is replaced. A convolutional head renders the model outputs directly into pixel frames instead of relying on a latent-to-pixel VAE decoder.
• Clean-video prediction is used. The authors report that predicting the clean video, rather than the diffusion velocity, is more effective for pixel-space training.
• A larger noise shift is introduced. Together with the prediction target, this scheduling change is identified as an important factor in maintaining quality.
• The model works at frame-level granularity. With the same number of tokens per frame, PixVid can produce video at roughly twice the resolution and can respond to a new action on the next frame.

Results and implications

Fine-tuned from Wan2.1-14B with approximately 1.3 million training clips, PixVid is reported to be competitive with its latent-space baseline. The supplied material also highlights depth-conditioned video editing: under fast motion, the generated edit remains aligned with the depth condition. In interactive settings, framewise operation reduces the need to wait for the completion of a larger temporal block.

The broader significance is that latent compression is being treated as a design choice rather than an unavoidable requirement. As video systems move toward higher resolutions, direct manipulation, and real-time control, pixel-space generation may offer finer temporal control and more immediate conditioning. PixVid is notable for making this transition while reusing a large pretrained diffusion backbone, which keeps the architectural change relatively focused.

The available material does not include detailed throughput, memory, perceptual-quality metrics, or broad model-by-model comparisons. It is therefore too early to conclude that pixel-space generation is universally more efficient. The durability of PixVid’s advantages across resolutions, video lengths, and hardware settings will require the full paper and independent reproduction.

Source: Hugging Face Daily Papers