ClawLoop Builds a Self-Improving Loop for Harness-Based Agents

Introduction

As language-model agents move from chat interfaces into sandboxes, filesystems, and shells, their training problem becomes broader than learning isolated tool calls. An agent must plan a sequence of operations, inspect intermediate state, recover from errors, and complete tasks in an open-ended workspace. Existing post-training pipelines for this setting are often one-shot: a stronger external model creates tasks and trajectories, the target model is trained once, and researchers inspect the evaluation results manually.

The paper ClawLoop proposes turning that pipeline into a recurring improvement cycle. Instead of treating evaluation as the endpoint, it uses the agent’s behavior to determine what should be trained next.

How the loop works

The framework has three connected stages:

• Task synthesis: ClawLoop uses two complementary construction routines to create a diverse pool of tasks. It then increases difficulty based on the agent’s diagnosis and applies both agent-free checks and agent-based validation to identify tasks that are unsuitable or not executable.
• Training: The resulting data can be used for supervised fine-tuning, reinforcement learning, or a combination of the two. The goal is to target weaknesses observed in the current iteration.
• Diagnosis and feedback: Rollouts are analyzed to produce a capability report. That report becomes input to the next task-synthesis round, closing the loop.

The distinction is important. This is not simply automatic prompt or benchmark generation. The proposed process attempts to make task difficulty and task selection responsive to concrete failure patterns. In principle, an agent that struggles with planning, environment inspection, or recovery should receive tasks designed to exercise those capabilities in the next iteration.

Reported results and limitations

The study uses Qwen3.6-27B as the model running the loop and reports consistent improvements on SkillsBench, Claw-Eval-Live, and ClawGym-Bench. The refined tasks are also transferred to smaller Qwen3.5-4B and 9B models, which show gains across the reported benchmarks. This transfer result suggests that the value of the generated data is not entirely tied to the scale of the model that created it.

The available material does not include exact benchmark scores, training costs, iteration counts, or a full comparison with alternative pipelines. It therefore does not establish how efficient the method is under different compute budgets. Nor does “self-improving” mean that all human design disappears: task validation, reward choices, and the reliability of diagnosis can still shape the outcome.

Why it matters

ClawLoop reframes harness-agent training as continuous data refinement rather than a single data-construction event. For agents that manipulate repositories, files, and command-line environments, failed trajectories can be useful training signals if they are categorized accurately. Evaluation becomes part of the data-generation machinery instead of merely producing a final score.

The approach also raises practical questions for future work. A convincing assessment will require transparent failure taxonomies, independent evaluation sets, and cost analysis across iterations. Otherwise, a loop may learn to perform well on its own synthesized tasks without demonstrating broader reliability. Even with that caveat, ClawLoop offers a clear template for connecting agent behavior, targeted task creation, and post-training in one harness-native process.

Source: Hugging Face Daily Papers