Chimera: A Hybrid Diffusion Backbone for Long-Context Visual Generation

Introduction

As visual generation moves toward higher-resolution images, longer videos, and richer multimodal conditioning, the cost of standard full-attention Transformers becomes a major bottleneck. Full attention is powerful, but its quadratic scaling with sequence length makes it increasingly difficult to use for long video diffusion models. Chimera, introduced by researchers including authors from Adobe, is presented as a hybrid visual diffusion backbone built specifically for this setting.

Key ideas

• One stream for text, images, and video: Chimera places text, image, and video tokens into a single raster-ordered sequence and does not rely on positional embeddings. The design aims to let the model learn multimodal and spatiotemporal structure through ordering and architecture rather than a separate position-coding mechanism.
• A hybrid alternative to full attention: Instead of applying full attention everywhere, Chimera combines several modules. Kimi Delta Attention (KDA) provides long-context state tracking with O(N) complexity. Interleaved Multi-head Latent Attention (MLA) keeps a path for direct global interaction. Modality-aware short convolutions then capture local spatial and temporal context.
• Sparse MoE for capacity without full activation cost: The architecture uses sparse Mixture-of-Experts layers to expand total capacity while limiting the amount of compute activated per token. The reported large model has 11B total parameters with 2B activated parameters.
• HeteroP for scaling a heterogeneous model: Because Chimera is not a uniform stack of identical Transformer blocks, simple width-and-depth scaling is not enough. HeteroP transfers hyperparameters module by module, based on each tensor’s functional fan-in and model depth. This creates a consistently tuned model family that can be used to fit Chinchilla-style compute-optimal laws.

Results and implications

According to the paper, when measured by pretraining diffusion loss, Chimera’s dense backbone is 1.7 times as compute-efficient as a matched full-attention Wan-2.1 2B baseline. The complete system reaches a reported 7.3 times improvement. The model also extrapolates from 5-second training clips to 30-second videos without length-specific fine-tuning, with only a 6.5% FID degradation in the final five seconds.

The broader message is that long-video diffusion may not be best served by simply scaling conventional full-attention Transformers. Chimera suggests a more modular path: use linear-time mechanisms for persistent state, occasional global attention for direct interaction, local convolutions for nearby visual structure, and sparse experts for capacity. Its fitted scaling laws also suggest that image and video pretraining may require different compute allocations across activated model size, token count, and data mix.

The available material is primarily an abstract-level summary, so details such as data composition, human preference results, qualitative samples, and deployment trade-offs require the full paper. Still, Chimera is a notable signal that visual diffusion backbones are moving toward more heterogeneous and compute-aware designs for long-context generation.

Source: Hugging Face Daily Papers