KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems

KVCMAS. Efficient KV cache Correction for Shared Context in Multi-Agent Systems. By Hyesung Jeon and four others, from Seoul National University. Published on the twenty-eighth of September. Its code repository is worked out: about seventy source files, with a readme.

Some multi-agent systems give every agent the same model and tell them apart only by their prompts, so that each plays a complementary role in solving a complex task. The catch is that each agent's own prefix changes the KV cache produced for the context they all share. As a result every agent prefills the growing context again and builds a separate cache, which costs a great deal of computation and memory. The text they share is identical, but because what comes before it differs, the cached state for that text differs too, and so none of it can simply be handed from one agent to the next.

Existing fixes fall short in different ways. Selective recomputation reduces the redundancy but still leaves substantial model execution. Methods based on delta correction either handle only context relations that recur, or keep memory-hungry online correction states when the context is changing. And when shared context is seen for the first time, these methods build a reference cache outside the agent workflow, while an approximate correction at the first agent then colours the outputs that every later agent receives.

KVCMAS is an online framework for correcting the KV cache. It describes how one agent's cache deviates from another's using compact low-rank states, and it chains those corrections along the agent workflow itself, so no extra reference prefill is needed. Instead of each agent building a whole cache of its own, or a reference cache being built off to the side, the difference between one agent's cache and the next is what gets stored and passed along, in compact form.

Two properties follow from that design. It supports shared context that keeps changing as the task goes on, which the earlier delta methods could only manage with memory-intensive online state, or not at all. And it keeps the first agent's cache exact, so the error that approximate methods introduce at the very start of the chain is not passed down to the agents that follow.

The authors test across several workloads, in language and in vision-language. KVCMAS matches or improves on the accuracy of earlier methods for sharing the KV cache, and it has the lowest time to first token under highly concurrent serving. On controlled serving traces, it is two point zero times faster to first token than inference that shares no cache at all. And it cuts peak GPU memory by up to three point seven times compared with an earlier method for KV cache correction.

The agents behave like a relay team passing one cache along. KVCMAS makes sure the first handoff is exact, and then carries small, low-rank corrections from runner to runner instead of recomputing the whole course each time. The authors present it as an accurate and scalable way to share the KV cache among agents that differ only in their prompts.