SoloQ Brings Calibration-Free Low-Bit Quantization to Diffusion LLMs

Introduction

Diffusion language models are emerging as an alternative to autoregressive generation. Rather than producing tokens strictly from left to right, they can update masked or noisy tokens through bidirectional denoising. This design creates new modeling possibilities, but it also introduces a demanding inference pattern. Full-sequence diffusion repeatedly invokes expensive forward passes, and block-diffusion systems add a persistent KV-cache that can become a significant memory burden as generation proceeds.

Low-bit weight-and-activation quantization is an obvious way to reduce this cost. Yet diffusion models make post-training quantization unusually difficult. Their activation distributions can shift with both the masking configuration and the denoising step. A calibration set collected from one portion of this process may therefore fail to represent other states. SoloQ addresses this problem by seeking a representation whose statistics are predictable enough to support quantization without calibration data.

What SoloQ does

• A normalized rotated basis: SoloQ transforms weights and activations into a normalized basis in which the marginal distribution after rotation is more predictable. Quantization parameters can consequently be designed independently of runtime samples.
• Structured K-RPBH rotation: The framework uses a structured K-RPBH rotation, paired with a lightweight rescaling correction. The combination is intended to make values easier to quantize while limiting the overhead of the transformation.
• Two deployment paths: A predictable post-rotation distribution enables both distribution-matched codebooks and hardware-native NVFP4. This gives the method a path toward specialized numerical representations as well as existing low-bit hardware support.
• KV-cache handling for block diffusion: Instead of quantizing the block that is still being actively denoised, SoloQ quantizes persistent KV states when a block is committed. The design aims to compress long-lived cache entries without perturbing the current denoising operation.

Reported results and significance

The paper evaluates SoloQ on full-sequence diffusion LLMs, including LLaDA and Dream, and on block-diffusion models such as Fast-dLLM v2 and Nemotron-Labs-Diffusion. According to the supplied material, the method retains accuracy under 4-bit quantization and outperforms the calibration-based baselines on knowledge- and reasoning-intensive benchmarks. With NVFP4, the authors report up to a 2.61x reduction in peak memory and up to a 2.24x end-to-end inference speedup.

The broader contribution is a shift in how quantization is approached for dLLMs. Conventional post-training methods often depend on representative inputs and carefully tuned ranges. That dependency becomes fragile when the model moves through many masking and denoising states. SoloQ instead uses a transformation with a designed statistical behavior, reducing the need to gather and maintain calibration data for every deployment condition.

The KV-cache component is also important. Quantization is not only a matter of compressing weights and activations inside matrix operations; persistent intermediate states can dominate memory in block-based generation. Separating the active block from committed cache entries provides a system-level strategy for reducing that pressure.

The supplied abstract does not provide every model, task, or hardware detail behind the reported gains, so practical results will still depend on implementation and deployment conditions. Even so, SoloQ presents a coherent route to low-bit diffusion inference by combining numerical transforms, cache management, and hardware-aware formats rather than treating quantization as an isolated weight-compression step.

Source: Hugging Face Daily Papers