JIT-Agent Makes Agent Harnesses an On-Demand Intelligence Layer

Introduction

Agent research often frames capability as a property of the underlying language model: a larger or better-trained model should plan more reliably, use tools more effectively, and complete longer tasks. JIT-Agent advances a different thesis. The runtime structure wrapped around the model—how it stores memory, decomposes goals, takes actions, and coordinates tools or skills—can be just as important, and may dominate performance in some settings.

The paper calls this surrounding software an agent harness. In current systems, harnesses are typically designed by engineers, tuned for individual tasks, and difficult to scale across models and domains. JIT-Agent is presented as a dedicated “harness intelligence” model that can synthesize a suitable harness on the fly for an off-the-shelf agentic LLM.

Key points

• A machine-generatable runtime artifact. The researchers formalize the harness through a fixed four-module protocol covering memory management, planning, action execution, and tool or skill orchestration. This gives the system a structured object that can be generated and recombined rather than a collection of ad hoc prompts.
• Task-level adaptation. Instead of applying one universal workflow, JIT-Agent is intended to inspect the task and produce a configuration suited to its requirements. The proposed design is model-agnostic in the sense that it can assist different existing agent models.
• Repair and evolution. The system can revise a harness when execution becomes unstable. It can also distill signals from an expanding archive of previous harness configurations, allowing successful patterns to inform later generation.
• Reported cross-model improvements. According to the abstract, DeepSeek-V4-Flash with JIT-Agent outperformed GPT-5.6 on DeepSearchQA and OdysseyBench by 9.1 and 4.3 points, respectively. GLM-5.2 reportedly gained as much as 20.2 points. The paper also claims that generated harnesses were competitive with mature runtimes such as OpenCode and Claude Code, and improved several scales of DeepSeek V4, Mimo-V2.5, and Qwen3.6.

Why it matters—and what remains unclear

The central implication is that scaling agents may not require replacing the base model every time. A better execution system could become an independent optimization axis, especially for search, coding, and long-horizon tasks. If harness strategies transfer between models and their useful configurations accumulate over time, agent development could gain a compounding layer of reusable engineering knowledge.

The available material is not enough to verify the full claim. It does not describe the four modules in implementation detail, the training data, inference cost, failure cases, or whether every comparison used identical budgets and model access. The reported margins therefore need to be read as claims from the paper rather than universal conclusions. Even so, JIT-Agent offers a clear research direction: treat the agent runtime itself as trainable, transferable, and evolvable intelligence rather than fixed application code.

Source: Hugging Face Daily Papers