CORAL-KV Uses Cross-Layer Prediction to Compress Video KV Caches

Introduction

For autoregressive video diffusion models, memory consumption grows with every generated frame. The attention key-value (KV) cache preserves historical information, but its expanding size can become a major obstacle to long-video generation. Compressing the cache helps control memory use, yet even small reconstruction errors may compound when later frames repeatedly attend to compressed history. The result can be declining visual fidelity or temporal discontinuities.

A paper on arXiv introduces CORAL-KV, a compression framework designed around a structural observation: cache states are not only redundant within an individual layer, but are also strongly related across layers.

How it works

Clustering-based methods generally group similar key and value vectors inside one layer and quantize the resulting residuals. CORAL-KV adds a cross-layer prediction stage. Cache states from one layer are used to predict keys and values in subsequent layers, and only the prediction residual is quantized. When the prediction is accurate, the quantity of information that must be stored can be reduced without directly encoding every original vector.

The method uses compact block-diagonal-plus-low-rank, or block-DPLR, predictors:

• Block-diagonal components model channel relationships within each attention head while keeping the predictor compact;
• A global low-rank component captures dependencies across attention heads that are not represented by independent head-wise modeling.

CORAL-KV also exposes a fidelity–memory trade-off. Predictors can be calibrated offline and reused across videos, reducing adaptation overhead. Alternatively, lightweight block-diagonal predictors can be refit for the current video when higher reconstruction fidelity is preferred.

Reported results

The authors evaluate the framework on HY-WorldPlay and MAGI-1. In the reusable configuration, CORAL-KV improves PSNR over QVG-4 by 0.32 dB on HY-WorldPlay and 4.95 dB on MAGI-1, while reducing cache memory by 48% and 37%, respectively. In the video-adapted configuration, it improves MAGI-1 PSNR over QVG-Pro by 3.77 dB and uses 33.7% less cache memory.

The paper also reports stronger reconstruction fidelity during extended generation and fewer temporal discontinuities, with generation latency remaining comparable to the evaluated alternatives. These figures reflect the paper’s experimental settings; practical gains may vary with model architecture, quantization choices, and video length.

Why it matters

The central contribution is a shift in how KV-cache redundancy is modeled. Instead of treating each layer as an isolated compression target, CORAL-KV uses the internal organization of the video model itself. Stable cross-layer dependencies could allow long-context video generation to operate under a smaller memory budget while limiting the propagation of compression errors.

Several deployment questions remain open. Reusable predictors may be more convenient, while per-video fitting may deliver better quality at the cost of additional calibration work. The real hardware cost of fitting predictors and quantizing residuals will also need to be measured across different serving systems. Still, the results suggest that cross-layer structure is a promising signal for scaling autoregressive video generation.

Source: arXiv