DeepSeek-V4.1-Flash: Rethinking KV Cache Compression for Long-Context Agents

Introduction

For long-context models, the main deployment challenge is no longer just whether a system can accept more tokens. It is also how those tokens are processed, retained, and moved between memory tiers. Long-horizon agents are particularly demanding because they may repeatedly revisit tool outputs, intermediate steps, and conversation history. Their workloads are often input-heavy, making prefill computation and KV cache management central cost drivers.

DeepSeek-V4.1-Flash is presented as a model-and-system response to that problem. The multimodal Mixture-of-Experts model has a 552B-parameter backbone and supports contexts of up to one million tokens. Its headline focus, however, is not parameter count. It is the reduction of KV cache pressure across both high-bandwidth memory and persistent storage.

Key points

• Different compute budgets for prefill and decode. The model uses a Causal Encoder-Decoder architecture. According to the paper, it activates 8B parameters per token during prefill and 16B during decode. This allocation is aimed at workloads where agents consume long inputs but generate comparatively shorter responses.
• Cross-layer reuse and low-precision caching. DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2, or CSA2, with FP4 KV caching. The reported global KV cache footprint, which remains in HBM, is 890 bytes per token—about one-quarter of the corresponding footprint of DeepSeek-V4-Flash.
• A separate strategy for persistent cache. Cache that is not continuously held in HBM may reside in SSD or host memory. The paper introduces SWA Bounded Replay as a deployment optimization and reports that it reduces persistent KV cache footprint to roughly one-eighth of the DeepSeek-V4-Flash baseline.
• Compression is paired with broader model changes. The material says the model delivers substantially better performance than the baseline despite its smaller cache footprint. It also describes streamlined architectural changes, multimodal pretraining on 45T tokens, and post-training for text-based and multimodal agentic scenarios.

Why it matters

The important idea is not simply to make one cache smaller. The paper treats long-context serving as a three-part systems problem: prefill computation, active KV state in HBM, and persistent state in SSD or host memory. Optimizing only one layer can move the bottleneck elsewhere. By combining an asymmetric compute design, attention-level reuse, reduced-precision storage, and a deployment policy, DeepSeek-V4.1-Flash illustrates a more integrated approach.

For agent platforms, cache size and transfer bandwidth affect how many sessions can be served concurrently and how much interaction history can remain available. Smaller cache footprints could also make long-running workflows easier to keep resident across memory tiers. If the reported ratios hold on production hardware and representative workloads, KV compression could become an important route to lower long-context serving costs.

The available material does not provide the full hardware setup, latency and throughput measurements, accuracy trade-offs, or task-by-task results. Those details are necessary before turning the reported cache reductions into a general deployment conclusion. The paper’s contribution is therefore best understood as a design direction and a set of reported results that still need to be examined in operational context.

Source: Hugging Face Daily Papers