LongHorizon-Harness reframes long-horizon agents as a state management problem

Introduction

As LLM agents move from short interactions to real-world workflows, reliability becomes less about a single correct response and more about staying on track across many dependent steps. A long-horizon agent may need to inspect an environment, call tools, revise an earlier decision, and keep a coherent view of what has already been completed. LongHorizon-Harness addresses a common failure mode in this setting: task execution, task state, and completion assessment are often mixed into one ever-growing context.

When state is buried in conversation history, it becomes difficult to know what the agent actually knows versus what it merely claimed. Even worse, a mistaken self-assessment can become part of the context and influence later decisions. This paper reframes long-horizon execution as a task-state management problem rather than simply a prompt or context-length problem.

Key ideas

• External task state: LongHorizon-Harness keeps task state outside the execution context, making progress and assumptions easier to track.
• Verified updates only: The state is updated with facts independently checked from the environment, not just with the executor’s own claims.
• Manage-Execute-Audit loop: A manager maintains the state and selects the next subtask. A fresh-context executor performs the subtask. A read-only auditor verifies the resulting environment state before the next round.
• Lower risk of error propagation: Since execution results are audited before they affect the global state, incorrect self-evaluations are less likely to steer the rest of the task.
• Backend flexibility: A lightweight AgentAdapter lets different model and harness backends be used without rewriting their native agent loops.

The reported benchmark results are notable. With LongHorizon-Harness, Qwen 3.7-Plus improves from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench 2.1, and from 2.8% to 8.3% on OSWorld 2.0. The framework also raises Claude Opus 4.7 from 20.0% to 34.3% on an OSWorld 2.0 subset.

Why it matters

The significance of this work is not only the benchmark lift, but the architectural lesson behind it. Long-horizon agents need a trustworthy operational memory. Simply expanding the context window does not automatically solve the problem if the contents of that context include unverified assumptions or mistaken progress reports.

LongHorizon-Harness adds a structure closer to a task ledger plus an audit layer. The executor can focus on the current step, the manager can preserve global progress, and the auditor can check whether the environment actually changed as expected. This separation may be especially important for terminal tasks, web interactions, and desktop environments where a single false belief can derail many later actions.

There are still practical questions beyond the provided material, such as overhead, latency, and how robust the auditor is in complex environments. Even so, the direction is clear: reliable agents will likely depend not only on stronger base models, but also on harnesses that manage state, verification, and recovery explicitly.

Source: Hugging Face Daily Papers