Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Vincent-Daniel Yun, Woosang Lim, Haneul Yoo, Sungjoo Yoo, Murali Annavaram, Sai Praneeth Karimireddy
University of Southern California · Seoul National University · New York University
Published September 29, 2026 · on the feed October 2, 2026 · ▲ 84 · arXiv 2609.32259
The abstract
Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared context already processed by the sender. Reusing the sender's key-value (KV) cache avoids this redundancy, but prefill-free transfer across model families must handle differences in tokenization, model depth, and KV representations. To address these issues, we propose HeteroFold, a prefill-free cross-family KV cache transfer method that keeps both the sender and receiver frozen. HeteroFold aligns model structures, maps the sender cache into the receiver space, and calibrates it to preserve receiver behavior. Across six transfer directions, HeteroFold achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings. It also matches text-based communication on the multi-agent benchmark. At 32K context length, Llama-3.1-8BrightarrowMinistral-3-14B transfer is 10.7times faster than Native Prefill and 1.18--1.47times faster than the state-of-the-art prefill-free baselines, Dense Latent and KV Ridge. These results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill.
The Brief
2:10A short podcast summary of the paper. Written and voiced by AI.
Transcript
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs. By Vincent-Daniel Yun and five others, from the University of Southern California, Seoul National University, and one other institution. Published on the twenty-ninth of September. No code repository is linked to it.
Multi-agent LLM systems increasingly mix different models, each chosen for a specialized role. When those agents talk in text, every receiver has to prefill the shared context again, even though the sender has already processed it. Handing over the sender's key-value cache would skip that repeated work. The obstacle is that models from different families do not line up. They tokenize differently, they have different depths, and their KV representations differ, so a cache built by one is not directly usable by another.
HeteroFold is a method for transferring the KV cache across model families without any prefill on the receiving side, and it keeps both the sender and the receiver frozen. Nothing is fine-tuned on either end.
It works in three moves, as the abstract lays them out. It aligns the structures of the two models. It maps the sender's cache into the receiver's space. Then it calibrates that mapped cache so the receiver keeps behaving the way it would on its own.
The evaluation covers six transfer directions. HeteroFold gets the best cache-transfer performance on all four long-context benchmarks and in most short-context settings, and on the multi-agent benchmark it matches communication through text. On speed, the reported case is a transfer from a Llama-3.1 sender into a Ministral-3-14B receiver at a context length of thirty-two thousand tokens. There it runs ten point seven times faster than native prefill. Against the strongest existing prefill-free baselines, Dense Latent and KV Ridge, it is between one point one eight and one point four seven times faster.
Keeping both models frozen is what makes the claim practical: a receiver can pick up another family's working memory without either model being retrained. Communication in text is the bar HeteroFold has to clear, and on the multi-agent benchmark it matches that bar, while skipping the prefill that text forces on every receiver.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Two models, one shared context, and the work done twice. That is the wall here.
You run a multi-agent system that mixes models from different families: one as solver, another as evaluator. They share a long context by sending text, so every receiver must prefill it again, though the sender already processed it.
The road ahead
- 1 · The problem ~7 min Agents from different model families redo work by re-reading shared text, but one model's memory of that text does not fit another model.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Line the two models up, map the sender's cache to the receiver's statistics, then tune it so the receiver attends and answers as it would natively.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min The method runs in three stages: align structure, map the cache into the receiver's space, then calibrate it against the receiver's native behaviour.
- 6 · The experiment ~6 min You’ll guess: At a 32K context length, how many times faster was HeteroFold's Llama-to-Ministral cache transfer than native prefill?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.