CUA-Master Uses Open-World Recursive Improvement to Scale Computer-Use Agents

Introduction

Building an agent that can operate a computer requires more than recognizing a screen and producing a click. A capable computer-use agent (CUA) must handle long workflows, changing interface states, failed actions, recovery, and outcome verification. CUA-Master addresses this problem by treating data construction and model optimization as a continuously improving loop rather than as a one-time trajectory collection process.

The proposed double loop

The paper presents a nested recursive improvement framework. At the system level, its outer loop is described as a world-data-model flywheel. Open-world tasks expose weaknesses in the current model; those weaknesses then guide the construction of new data and the next optimization cycle. In principle, this allows the training distribution to evolve with the agent instead of remaining fixed around the capabilities it already has.

The inner loop focuses on data production and combines two mechanisms:

• Open-world seeding: authentic workflows are mined from public instructional videos and organized into seed tasks. CUA-Master reconstructs interaction trajectories through both visual and semantic paths, reducing reliance on surface-level visual imitation alone.
• Autocatalytic propagation: seed tasks are converted into reusable action primitives. These primitives can then be recombined and transformed to synthesize tasks that are more varied, more complex, and easier to verify.

The design is important because real demonstrations are expensive and usually cover only a narrow slice of possible computer workflows. Reusable primitives provide a way to expand that coverage while preserving some control over task structure and expected outcomes. The framework does not simply ask the model to generate arbitrary experiences; it starts from open-world examples and uses them as building blocks for controlled synthesis.

Making trajectory collection more reliable

CUA-Master also introduces a feedback-guided snapshot-and-rollback retry mechanism for interaction in real environments. When an action fails, the system can use environmental feedback to return to a suitable earlier state and retry, rather than discarding an entire trajectory after a local error. According to the supplied material, this mechanism is intended to improve the efficiency and reliability of trajectory collection; implementation details are not included in the excerpt.

Results and implications

The reported CUA-Master-27B scores are 80.2 on OSWorld Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench, and 25.0 on Agents' Last Exam. These evaluations cover GUI-only as well as GUI-CLI hybrid settings. The authors also report continued gains in data-synthesis quality and efficiency across recursive iterations.

The broader message is that scaling computer-use agents may depend on the quality of the improvement loop as much as on model size. Open-world data supplies new task distributions, model failures reveal missing capabilities, primitive-based synthesis expands coverage, and rollback reduces the cost of collecting usable trajectories. At the same time, the supplied material contains mainly the abstract and headline metrics. Detailed ablations, data volumes, and generalization limits require examination of the full paper before stronger conclusions can be drawn.

Source: Hugging Face Daily Papers