StarHarness Evolves Enterprise Agents Without Changing Model Weights

Enterprise agents often fail for reasons that are not captured by model-scale comparisons. A model may be capable of reasoning about a task, yet still struggle because a tool schema is ambiguous, an operational convention is undocumented, or the execution loop encourages the wrong sequence of actions. StarHarness focuses on this surrounding layer: the agent harness.

What the framework changes

StarHarness keeps the underlying model weights fixed and searches for a harness better suited to a particular environment. The search space extends well beyond system prompts. It can include task framing, tool interfaces and schemas, reusable skills, MCP-backed providers, subagent organization, state handling, context management, and the configuration of the agent loop.

This makes the harness an explicit engineering object. Instead of asking only whether a model is capable, the approach also asks whether the model is being given the right interfaces, operational knowledge, and control flow to use that capability.

The search process has three notable safeguards:

• Failure-based stratification: tasks are grouped by how the default harness fails, creating a compact evolution pool that represents different failure modes.
• Separated proposal and selection: proposer-visible search tasks are kept distinct from hidden selection tasks, making it harder to optimize narrowly for the examples used to generate changes.
• Held-out evaluation: tasks excluded from evolution are reserved to test whether improvements generalize beyond the search pool.

Reported results

The evaluation covers ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance. According to the paper’s summary, after roughly four to twelve accepted changes per environment, evolved harnesses improve full-benchmark performance by about 20 to 35 percentage points over the default harness. The gains persist on tasks excluded from evolution and transfer across GPT and Qwen model families without re-evolution. The accompanying project description also reports inference-cost reductions of 17% to 53% in the evaluated settings.

One striking comparison comes from ITBench: Qwen3.5-27B with an evolved harness is reported to outperform GPT-5.5 running the baseline harness by 19.2 points. This should not be read as evidence that smaller models are generally stronger. It is better understood as evidence that the quality of the model-environment interface can materially affect measured agent capability.

Why it matters

Trace analysis associates the improvements with repaired interfaces, environment conventions, and operational knowledge that shorten the search for a workable solution. Several settings also show fewer false-positive diagnoses and shorter trajectories. For enterprise deployments, this points to an alternative to immediately replacing a model: improve the tools, schemas, context policy, and execution control around it.

The available material does not establish the cost of the evolution process, the independent contribution of each harness component, or the limits of transfer across unrelated environments. StarHarness is therefore best viewed as an environment-specific optimization method, not a substitute for model training or broad agent evaluation.

Source: Hugging Face Daily Papers