Self-HPO Lets Coding Agents Improve Both Their Harness and Policy

Introduction

Coding-agent performance is often discussed as if it were mainly a property of the underlying language model. In software engineering tasks, however, the model’s surrounding harness can be just as important. The harness determines how an agent inspects a repository, invokes tools, runs tests, and responds to feedback. A policy that is strong in one interaction setup may therefore perform poorly when the setup changes.

Self-HPO, proposed by a team including researchers from Microsoft Research, treats the policy and the harness as mutually dependent parts of one learning system. Instead of optimizing only the model or freezing the interaction workflow, it alternates improvements to both.

How the loop works

The framework has two recurring stages:

• H-step: harness search. The current policy examines its own failures, identifies likely failure modes, and proposes new harness candidates. The search is targeted at the tasks the current policy struggles with rather than being a general redesign of the entire agent stack.
• P-step: policy optimization. The policy is updated with reinforcement learning under the selected harness. The resulting trajectories provide a different and potentially more useful training environment than a fixed harness would offer.
• Hard-task transfer. Difficult tasks encountered during training become the search set for the next harness round. As the policy evolves, the harness search follows its changing weaknesses instead of remaining tuned to an earlier checkpoint.

This addresses two forms of mismatch. A harness tuned for an older policy may fail to expose the weaknesses of a newer one, while policy training under a fixed harness can only learn from the behaviors that this harness makes possible.

Results and findings

Across SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro, Self-HPO outperformed baselines that optimized only the harness or only the policy. For Qwen3.5-35B-A3B, the reported average resolve rate increased from 57.8% to 64.8%. The gains also generalized to out-of-distribution settings, suggesting that the method was not limited to the exact tasks used during optimization.

The component analysis is particularly notable. An optimized harness still retained benefits when paired with the base policy, and an optimized policy retained benefits with the base harness. Yet combining both produced a gain larger than the sum of their individual improvements. This points to genuine interaction between the agent’s learned behavior and the environment in which that behavior is trained.

Self-HPO also improved Pass@1, a measure more closely related to single-attempt reliability, and expanded the capability frontier reflected by Pass@k. The framework is designed to operate without an external model proposing harnesses. The paper additionally reports further improvement when a stronger external model, such as Opus 4.8, is used as the harness proposer.

Why it matters

The broader contribution is to make the coding workflow itself a learning target. In conventional systems, engineers hand-design the tool loop and then train or prompt the model inside that fixed structure. Self-HPO instead lets observed failures influence environment design, which then changes the data available for policy learning.

This could be useful as coding agents encounter new repositories, tools, and failure patterns. Still, the available material does not specify the computational cost of repeated search, the precise reinforcement-learning configuration, or long-term behavior over many optimization rounds. Self-HPO should therefore be viewed as a promising co-optimization framework, not yet as a complete solution to continual improvement for coding agents.

Source: Hugging Face Daily Papers