DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

DeepSeek-V4.1-Flash. Pushing the Limits of KV Cache Compression. By DeepSeek-AI and five hundred and ninety-one others, from DeepSeek AI. Published on the seventeenth of September. No code repository is linked to it, though it does publish its weights.

Long-horizon agents have made model workloads increasingly heavy on input. Earlier work has cut the cost of long-context computation substantially, but prefill is still computationally expensive. Large KV caches also strain the capacity of HBM and SSD storage, and the bandwidth for moving data between them. Together, those demands on compute, storage and bandwidth are the main obstacle to making deployment cheaper. When a workload is dominated by input, the costs that matter most are reading that input once, in prefill, and holding what was read, in the cache.

DeepSeek-V4.1-Flash is a multimodal Mixture-of-Experts model with five hundred and fifty-two billion backbone parameters, supporting contexts of up to one million tokens. Its Causal Encoder-Decoder architecture activates sixteen billion parameters per token during decode, but only eight billion during prefill. For input-heavy agentic workloads, that makes the expensive part cheaper. Prefill is where the model reads its input, so activating fewer parameters there goes straight at the compute side of the bottleneck.

The KV cache is compressed from two directions. Compressed Sparse Attention 2 reuses KV cache across layers, and the cache is stored in FP4. Together these bring the global KV cache footprint, which always sits in HBM, down to eight hundred and ninety bytes per token. That is roughly a quarter of the footprint of DeepSeek-V4-Flash. Reusing the cache across layers means fewer distinct entries to keep, and storing them in FP4 means each entry is smaller.

A dedicated deployment optimization called SWA Bounded Replay then reduces the persistent KV cache footprint, which always sits on SSD or in host memory, to roughly an eighth of DeepSeek-V4-Flash's. The architecture of DeepSeek-V4 is also streamlined, with several efficient architectural extensions added. Two footprints, then, each reduced separately: the one that has to live in the fastest memory, and the persistent one kept further out.

Despite the much smaller cache, the model performs substantially better than the baseline. The two results are stated together: a much smaller cache, and substantially better performance than the baseline, rather than one bought at the cost of the other. It was pretrained on a multimodal corpus of forty-five trillion tokens, followed by comprehensive post-training, and it shows strong performance across diverse agentic scenarios, both text-based and multimodal. Model checkpoints are available on Hugging Face.

The report treats memory as the constraint to design against. Fewer active parameters during prefill, a smaller cache in fast memory, and a smaller one in slow memory all aim at the same bill: what it costs to serve an agent that reads far more than it writes. The context length makes this pressing. With up to one million tokens in context, every byte saved per token is saved a million times over.

DeepSeek-V4.1-Flash. Pushing the Limits of KV Cache Compression. By DeepSeek-AI and five hundred and ninety-one others, from DeepSeek AI. Published on the seventeenth of September. This is the long version: the whole paper rather than the abstract.

Long-horizon agents have made model workloads input-heavy. Every tool call sends a new prompt, and when the cached state for that prompt is missing the whole prefix must be processed again. Sparse attention has already cut the computing cost of long contexts, so the bottleneck has moved to memory: the key-value cache held in fast GPU memory while a request runs, the persistent copy kept on SSD and in host memory for reuse, and the bandwidth to move it around. DeepSeek-V4.1-Flash is built to shrink that cache. It is a multimodal mixture-of-experts model with five hundred and fifty-two billion backbone parameters and a context of up to one million tokens, activating eight billion parameters per token during prefill and sixteen billion during decode.

The starting point is the view that DeepSeek-V4 is really a local model with compressed global context attached. Each layer has a sliding window of recent tokens, only one hundred and twenty-eight here, whose cache stays the same size however long the input grows. The global branch, which can reach the whole context, is what grows. So the effort goes into the global branch. The paper names three multiplicative ways to shrink it: make each cache entry smaller, compress several tokens into one entry, and let layers share entries instead of keeping their own. Earlier work on sharing across layers covered only parts of this, and none covered all three dimensions.

Four moves do the work. First, the Causal Encoder-Decoder splits forty layers into a twenty-layer encoder and a twenty-layer decoder. The decoder's global cache is projected directly from the encoder's final hidden state, so prefill only has to run the encoder to build it, which nearly halves prefill computation. Second, Compressed Sparse Attention 2 gives every layer one of three fixed modes. A Full layer computes its own global cache and picks its own top entries to attend to. A Reindex layer borrows an earlier layer's cache but rescores it with its own query. A Reuse layer borrows both the cache and the selection. In the encoder, each group of six layers has one Full layer and five Reuse layers; in the decoder, only a single layer ever computes a global cache. Third, a hierarchical indexer lets that first decoder layer nominate up to sixteen thousand three hundred and eighty-four candidate positions, and later indexers search only those, so their cost no longer grows with context length.

Fourth, the global cache is stored in four-bit floating point. The authors drop the usual second-level scale and argue from the architecture that it is safe: after normalization the cached values are bounded near twenty-two point six, and the largest seen in training was around ten, far inside the format's range. The sliding-window cache stays at eight bits because it is more sensitive. Together this brings the global cache to eight hundred and ninety bytes per token, about a quarter of DeepSeek-V4-Flash.

The persistent cache gets a deployment trick. In V4, sliding-window state took nearly half of it, yet that state dies within minutes while global state keeps being reused and is retained for at least seventy-two hours. Rebuilding it exactly means replaying the window once per layer, because the dependencies stack. SWA Bounded Replay replays only the last window of tokens and accepts an approximate state. That lets window state leave the persistent cache for a small, short-lived pool in host memory. Halving the persistent cache that way, on top of the quartered global cache, gives the one eighth figure.

The base model is compared with DeepSeek-V4-Pro-Base, which has three times the total parameters. It matches or beats Pro on MMLU-Pro and HumanEval, but trails it clearly on SimpleQA-Verified. After post-training, DeepSWE rises to seventy-four point two from V4-Flash's fifty-four point four, and Terminal-Bench 2.1 reaches ninety point six. Decode computation grows by only a quarter as context grows two hundred and fifty-six fold.

The limitations share a closing section with the conclusion, and they are candid. The new architecture creates robustness boundaries that are not yet fully characterized. Wrong selections by the sparse attention, and the approximate reconstruction in bounded replay, may still degrade the model in untested edge cases, and the authors single out sparse retrieval over long contexts and state reconstruction at cache resumption points for further stress testing. They also say benchmark parity with closed frontier models does not mean matching them on the hardest reasoning. Terminal-Bench 4.0 shows it: thirty-one point two against fifty-one point eight for Opus-5. And they state plainly that post-training added no algorithmic novelty; the gains came from data and environments.

The idea to carry away is a trade between storing and recomputing. Global state is expensive to rebuild and reused over long periods, so it is compressed hard and kept. Window state is cheap to approximate and dead within minutes, so it is thrown away and roughly rebuilt on demand. Compression here is architecture, number format and deployment policy designed together.