Researchers Build Cache-to-Cache Protocol That Makes AI Model Handoffs 14 Times Faster

Quick Facts

• C2C reduces multi-model relay latency by roughly 14x in tests, dropping one step from 1,312 milliseconds to approximately 90 milliseconds.
• The method improves accuracy 3.1 to 5.4 percentage points over text-based model communication and 6.4 to 14.2% over individual models.
• The paper was accepted at ICLR 2026 and the team plans to release an agent-managed KV-cache serving system in September 2026.

Researchers at Tsinghua University's NICS-EFC lab, working with Shanghai AI Lab and Infinigence AI, have published a method that lets large language models communicate through their internal memory states rather than generated text. The approach, called Cache-to-Cache, or C2C, was accepted at ICLR 2026 after first appearing as an arXiv preprint in October 2025.

The problem C2C targets is straightforward. When one AI model hands off to another in a multi-model pipeline, it must write its internal reasoning out as text tokens, word by word. The receiving model then reads those tokens and rebuilds context before generating its own response. That sequential process adds latency at every step and discards much of the semantic information the first model accumulated.

How the System Works

C2C replaces text handoffs with direct transfers of KV-cache states, the internal memory structures models build as they process information. One model acts as the Sharer, the other as the Receiver. A small neural component called the Cache Fuser projects the Sharer's KV-cache into the Receiver's representation space and fuses it with the Receiver's own cache through a residual connection.

The researchers froze both language models during training and only trained the Cache Fuser bridge, keeping compute requirements low. A learnable gating mechanism selects which layers in the Receiver benefit from the incoming cache data.

The Cache Fuser does not expand the size of the KV-cache, meaning teams get richer context transfer without additional storage overhead. The Python package is named Rosetta, after the artifact that enabled translation between scripts.

Performance Numbers

In experiments, replacing 80 generated communication tokens with a direct cache transfer cut one relay step from 1,312 milliseconds to roughly 90 milliseconds. Across tested model pairs, C2C delivered an average 2.5x speedup in latency.

Accuracy results were also positive. C2C outperformed text-to-text communication by 3.1 to 5.4 percentage points and outperformed individual models by 6.4 to 14.2%. Fusing both models' KV-caches using the residual connection, rather than discarding the Receiver's cache entirely, increased accuracy by 24.18 percentage points over pure projection.

The team tested combinations spanning different model families, including Qwen, Llama, and Gemma, different sizes from 0.6 billion to 14 billion parameters, and different specializations covering general, code, and math models. They also tested across training stages, mixing pretrained and instruction fine-tuned models.

What This Means for Enterprise AI Teams

For companies building multi-agent systems or model routers, the research points to the communication layer as an untapped optimization target. Most current systems treat model handoffs as a fixed cost. C2C suggests that cost can be substantially reduced.

The researchers outlined several directions they see as viable next steps. These include using C2C as a communication primitive in agentic systems with multi-round reasoning and tool use, extending cache fusion to vision-language models, and applying the method to speculative decoding for lower inference costs.

There is also a privacy angle. Transferring KV-cache segments rather than explicit text limits what content is exposed during model-to-model communication, which could matter for enterprises handling sensitive data in agent pipelines.

The team noted that an agent company has already requested APIs to identify, compress, and transfer KV-cache states across nodes, a sign that production demand for this capability is forming ahead of the tooling.

A key constraint remains: C2C requires direct access to model internals. The method cannot be applied to closed, API-only models where KV-cache states are not exposed. Teams using proprietary model APIs from major providers would need those providers to support cache-level access before C2C-style communication becomes viable in their stacks.

The team plans to release an agent-managed KV-cache implementation alongside a serving system in September 2026.

Read more: Text handoffs slow AI models down. C2C lets them communicate through KV caches instead