Context Language Models Let Models Manage Their Own Context

Introduction

A larger context window does not automatically produce better memory. For agents that browse, code, or collaborate over long periods, the harder problem is deciding what should remain available, what can be compressed, and what should be removed. Most current systems delegate these decisions to an external harness, summarizer, retriever, or hand-designed policy. Context Language Models, or CLMs, take a different approach: they make the language model responsible for managing its own working context.

Treating context as a file

The central abstraction is simple. CLMs represent context as a file and allow the model to make unrestricted updates to it. The model does not merely append new observations or retrieve old text. It can reorganize, rewrite, and maintain the file as a task develops. Context management therefore becomes part of model behavior rather than a fixed combination of truncation, summarization, and retrieval rules.

The same abstraction extends to multi-agent settings. Each agent can maintain its own context file, allowing separate task states and more explicit information boundaries. It also creates a path for learning context-management strategies through in-context behavior or model parameters, instead of encoding every decision in an external controller.

Reported results and optimization

According to the paper’s abstract, zero-shot CLMs built with existing models outperform state-of-the-art context-management strategies on several evaluations:

• BrowseComp-Plus shows 11.4% higher accuracy with 21.5% fewer FLOPs;
• 12-hour EdgeBench shows a 5% score improvement with 59% fewer FLOPs;
• a 24-hour multi-repository agent-swarm task reports 65% greater improvement at the same compute budget.

The authors also examine how the design can be improved. A standard skill-optimization loop can evolve natural-language instructions for context management, raising held-out accuracy by up to 35.9 points on a context-management task while reducing compute. An online reinforcement-learning method improves Qwen3.5-9B on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs.

Serving efficiency is addressed as well. The proposed Suffix Cache Reuse mechanism is co-designed for CLMs and reportedly reduces server-side compute by a further 35% compared with standard SGLang at matched performance. This suggests that giving a model more freedom over its context does not necessarily require proportional increases in inference cost.

Why it matters

CLMs shift the boundary between the model and the surrounding agent framework. Instead of forcing the harness to define a universal memory policy, the model can adapt its context organization to the task. That could provide a more unified foundation for long-horizon agents, coding systems, and multi-agent workflows.

The available material does not include the full benchmark configurations, the precise file-update mechanism, or the error distribution across tasks. The results should therefore be read as evidence for a promising systems pattern, not as proof that long-term memory, reliability, or controllability has been solved. Preventing important information from being deleted, auditing context changes, and maintaining stable behavior in more complex environments remain open questions.

Source: Hugging Face Daily Papers