Harness² Helps Agents Improve Without Ground-Truth Verifiers

Introduction

An agent’s performance depends on more than the language model behind it. The way it plans a task, selects tools, checks intermediate results, and recovers from failure is equally important. This surrounding workflow—often called an execution harness—can determine whether a capable model succeeds in a real work setting. Google’s Harness² explores how such harnesses can improve recursively, with a particular focus on open-world tasks where reliable answers or verifiers may not exist in advance.

A two-level improvement loop

Many recursive self-improvement systems rely on rewards, tests, or ground-truth evaluators. That assumption becomes fragile when tasks are unseen during harness design and novel at test time. Harness² addresses the problem through two connected forms of recursion:

• Task-level recursion adapts the harness to the task at hand. The system proposes alternative edits, probes their behavior, and composes a stronger version through a contrastive Propose-Probe-Compose procedure.
• Harnessing-level recursion extracts reusable lessons from those edits and updates a continual improvement harness, allowing later tasks to benefit from earlier adaptation.
• Scalable recursion supports parallel or sequential task-level refinement, so additional budget can be used to explore more candidate changes.

What the reported results show

The paper evaluates the approach on three professional benchmarks and four model-harness configurations. A single recursion step consistently improves GLM-5 and Gemini 3.5 Flash over their respective base harnesses. The reported maximum gains reach 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB. The supplied results also indicate that increasing the task-level recursion budget can produce further gains.

Why it matters

The important shift is conceptual: agent improvement does not have to mean only training a larger model or designing a better prompt once. Harness² treats the execution process itself as an evolving component. A useful adjustment discovered while solving one task can become a starting point for future work, which is particularly relevant to tool-heavy and open-ended professional environments.

The approach also has clear trade-offs. Probing and comparing edits consume extra inference and execution resources, while a preference between candidate behaviors is not the same as a verified task outcome. The remaining questions include how reliably experience transfers across domains, how costs scale with recursion depth, and whether poor strategies can accumulate over time.

Source: Hugging Face Daily Papers