Tools Do Not Have to Speak Text: Coca-Cola Compresses Agent Context into Latent Tokens

Introduction

For a multi-step language-model agent, the cost of tool use is not limited to the tool call itself. Search results, page excerpts, and intermediate observations are often appended to the context in raw text form. As the interaction continues, the agent must sift through an expanding mixture of relevant facts, repeated passages, formatting, and noise. The result is higher context consumption and a greater risk that attention will be diluted.

A team from Tsinghua University proposes Coca-Cola, short for Context Compression via Continuous Latent Tokens, as an alternative to this text-centric interface. Rather than requiring the agent to read the full tool output, the system compresses the observation into continuous latent tokens and places those representations in the agent’s input-embedding space.

The central idea: preserve decisions, not prose

Existing pruning and summarization methods remain tied to discrete text. They may remove tokens or generate a shorter textual summary, but the output still has to obey the constraints of language-based representation. Coca-Cola takes a different route. It does not attempt to reconstruct the original observation or produce a human-readable account of everything that happened. Instead, it learns a compact representation that retains information sufficient for the agent’s next action.

The paper’s main training mechanism is Action-Preserving Training. The compressor observes the same tool result that the full agent receives, while the training objective compares the agent’s next-action distribution under the original observation with its distribution under the compressed one. The target agent remains frozen, so the compressor is trained to approximate the decision-relevant information already used by that agent.

This changes the optimization target. A good compressed representation does not need to preserve every sentence, citation format, or repeated search snippet. It needs to preserve the distinctions that affect whether the agent searches again, follows a particular lead, performs another operation, or produces an answer.

Results and limitations

Coca-Cola is evaluated with web-search agents on Natural Questions, TriviaQA, 2WikiMultiHopQA, and HotpotQA. The experiments cover two model families, Qwen and GLM. According to the reported results, total agent token consumption falls by 70.01% to 89.24%, while end-to-end runtime decreases by 24.45% to 55.97%. Answer exact match and F1 are higher than those of the uncompressed agent in every evaluated setting.

These findings suggest that reducing context can improve rather than merely preserve performance when the removed material is mostly distracting. They should not, however, be read as evidence that the method is already general across all agent workloads. The reported evaluation focuses on web-search question answering and two model families. Code execution, database interaction, long-horizon planning, and multimodal tools may expose different failure modes.

Latent observations also create practical questions. They may be efficient for the model while being difficult for a person to inspect. If the compressor drops a crucial detail, debugging the failure could be harder than reading a conventional summary. Reliability, interpretability, transfer across tasks, and security auditing will therefore matter alongside compression ratios.

Why it matters

Coca-Cola treats tool output as an interface for decision-making rather than as text that must remain readable. That is a meaningful shift in agent design: tools can communicate through task-oriented representations tailored to the downstream policy. If the approach transfers beyond the current benchmarks, agents could support longer workflows with less context and inference overhead.

The authors have released code, creating an opportunity to test the idea in other tool-use settings and to study how much transparency can be retained while using continuous latent communication.

Source: Hugging Face Daily Papers