CacheBack Sends Multi-Agent Systems Only the KV Cache They Need

Multi-agent systems do not merely need a way to exchange information; they need a way to avoid exchanging information that is irrelevant to the next agent’s task. A sender may have processed a large context, while the receiver needs only a few pieces of evidence from it. Passing everything can therefore turn communication into the main memory and latency bottleneck.

Beyond text messages

The conventional solution is to ask the sender to generate a textual summary. Text is compact and relatively easy to inspect, but it requires decoding and may omit details that the receiving agent needs. Latent communication takes a different route by transferring model-internal states, particularly key-value caches, instead of requiring a new text message for every handoff.

This approach can avoid part of the generation cost and preserve information that a summary might lose. It also creates a scaling problem. A full KV cache grows with the context processed by each agent and with the number of agents participating in the workflow. Those caches can quickly exceed practical GPU-memory budgets or the receiving model’s context capacity.

Conditioning communication on the receiver

The paper’s central observation is that the sender does not need to transmit everything it knows. The receiver should first communicate a compact description of what it needs for its local task. The sender can then use that description to filter and compress the states it is about to transfer.

CacheBack is presented as a simple, training-free implementation of this idea. It relies on the sender’s attention weights as a relevance signal, selecting the portions of the KV cache that are more likely to matter to the receiver. The method does not introduce a separately trained communication model. Instead, it changes the criterion for transfer: the question is no longer what the sender happened to process, but which parts are useful for the receiver’s next step.

Reported results and practical implications

On FanOutQA, CacheBack with Qwen 3 removes 75% of the state that would otherwise be received. The paper reports a 14.7-percentage-point accuracy improvement and a 3.2x reduction in median task-completion latency relative to text communication. Similar improvements are reported across dense Transformers, Mamba-attention hybrids, and sliding-window attention models, suggesting that the basic idea is not tied to one architectural family.

The broader lesson is a useful systems principle for agentic inference: communication should be task-conditioned rather than automatically complete. Receiver-aware selection could reduce memory traffic, context pressure, and waiting time in workflows where agents repeatedly exchange large intermediate states. It may be particularly relevant for fan-out or specialist-agent designs.

The available material does not establish the method’s full behavior across compression ratios, tasks, or heterogeneous model combinations. Deployment would still need to address cache compatibility, errors caused by relevance filtering, and the engineering cost of integrating latent state exchange. Even so, CacheBack offers a relatively lightweight direction for making KV-cache-based collaboration more scalable.

Source: Hugging Face Daily Papers