Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models
Rubin Wei, Siyuan Sun, Hao Sun, Yuliang Liu, Xiang Shi, Jiarui Wang, Junming Zhang, Zhouhan Lin
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
Published October 18, 2026 · on the feed October 23, 2026 · ▲ 60 · arXiv 2610.25100
The abstract
Large language models need effective ways to acquire and use external knowledge beyond what they retain from pretraining. Continued pretraining risks catastrophic forgetting, while retrieval at inference time can add substantial overhead. External parametric memory avoids online retrieval, but constructing its training data requires extensive offline search, and output-level interpolation provides only shallow fusion. We introduce Deep Fusion Memory (DFM), a parametric memory architecture that reduces this construction workload and incorporates learned memory into intermediate hidden states. A causal memory predictor learns to generate memory from the preceding context, using retrieved chunk embeddings from the corpus as supervision. The predicted memory is shared across tokens within each chunk and fused into their hidden states through gated cross-attention. Memory prediction and fusion are trained end to end with the backbone frozen, and inference requires no explicit retrieval. Compared with existing parametric memory approaches, which often construct token-level memory targets, DFM requires 32times fewer retrieval queries and 2048times fewer retrieved items to construct its training data. Across six QA tasks, DFM performs better over the strongest baseline by 12.71% on Mistral-7B-v0.3 and 11.19% on Qwen3-8B-Base. Beyond QA, domain adaptation yields average gains of 6.51 and 4.38 points over Qwen3-4B-Base and Qwen3-8B-Base, respectively.
The Brief
2:39A short podcast summary of the paper. Written and voiced by AI.
Transcript
Deep Fusion Memory. A Chunk-Level Parametric Memory for Large Language Models. By Rubin Wei and seven others, from Shanghai Jiao Tong University and the Shanghai Artificial Intelligence Laboratory. Published on the eighteenth of October. No code repository is linked to it.
Large language models need effective ways to acquire and use external knowledge, beyond whatever they retained from pretraining, and each of the existing routes charges a price. Continued pretraining risks catastrophic forgetting. Retrieval at inference time can add substantial overhead to every query. External parametric memory sidesteps online retrieval, but it has two weaknesses of its own. Constructing its training data requires extensive offline search, and combining it with the model through output-level interpolation provides only shallow fusion: the memory meets the model at the very end, rather than inside it.
Deep Fusion Memory, or DFM, is a parametric memory architecture built against both weaknesses. It reduces the workload of constructing training data, and it incorporates learned memory into the model's intermediate hidden states, which is where the word deep in its name comes from.
The central piece is a causal memory predictor. It learns to generate memory from the preceding context, and the supervision it learns from is retrieved chunk embeddings taken from the corpus. Memory then works at the level of chunks. The predicted memory is shared across all the tokens within a chunk, and fused into their hidden states through gated cross-attention. Prediction and fusion are trained end to end while the backbone stays frozen. Once trained, inference requires no explicit retrieval at all: the predictor stands in for the lookup.
The first payoff is in building the training data. Existing parametric memory approaches often construct token-level memory targets. Compared with them, DFM needs thirty-two times fewer retrieval queries, and two thousand and forty-eight times fewer retrieved items, to construct its training data.
Quality improves as well. Across six QA tasks, DFM performs better than the strongest baseline by twelve point seven one percent on Mistral-7B-v0.3, and by eleven point one nine percent on Qwen3-8B-Base. The gains extend beyond question answering. In domain adaptation, DFM yields average gains of six point five one points over Qwen3-4B-Base, and four point three eight points over Qwen3-8B-Base.
Underneath the numbers is a swap of search for prediction. Rather than looking knowledge up while answering, or retrieving densely just to build token-level targets, DFM learns to anticipate the memory a chunk needs from the context before it, and feeds that memory straight into the hidden states of a model whose own weights never move.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Models forget when retrained and slow down when they search. Here is the bind.
You want a language model to use knowledge from a new corpus. Training it further on the corpus risks wiping out what it already knows.
The road ahead
- 1 · The problem ~6 min Every existing way to give a language model new corpus knowledge pays a price: forgetting, slow answers, or costly training data and a shallow hookup.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Learn one memory per chunk from the text so far, supervised by one lookup per chunk, and inject it into the model's middle layers.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Retrieve chunk targets offline, predict each chunk's memory from earlier chunks, share it across the chunk's tokens, and fuse it through gated cross-attention.
- 6 · The experiment ~6 min You’ll guess: Compared with token-level memory approaches, how many times fewer retrieval queries does DFM need to build its training data?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.