MAI-Gym Unifies GUI Agent Evaluation, Data Generation, and RL Training

Introduction

A GUI agent must do more than produce a plausible textual answer. It has to inspect an interface, choose an action, react to changing state, and determine whether the requested outcome was actually achieved. This makes online reinforcement learning for GUI use substantially more expensive than training on many mathematics or coding tasks: environments must be launched, interaction state must be preserved, and success must be verified. MAI-Gym, introduced by the Tongyi-MAI team, addresses this challenge by combining infrastructure, task synthesis, evaluation, and training in one open platform.

One workflow for GUI agents

MAI-Gym places evaluation, data collection, and online RL on a shared rollout pipeline. The same execution path can be used during training and benchmarking, reducing inconsistencies between the environment in which an agent learns and the environment in which it is measured. The platform covers mobile, computer, and web settings, supports ten established benchmarks, and is designed to scale to as many as 20,000 parallel environment instances.

The platform also treats environment and task creation as first-class components. It can synthesize resettable mocked applications, coherent tasks spanning multiple applications, and code-based verifiers that check whether an execution actually satisfies the task. The release includes 49 simulated apps, 2,000 execution-verified tasks, and reusable skills for app and task synthesis. Such verifiers provide a clearer reward signal for online learning than subjective manual inspection.

Results from MAI-Gym-9B

Using the platform, the researchers trained MAI-Gym-9B, a unified 9-billion-parameter agent for mobile and computer use. Among open GUI agents with fewer than 10 billion parameters, it reports 78.4% on MobileWorld and 76.2% on OSWorld-Verified. These results suggest that a common training setup spanning device types can support capable operation across more than one GUI domain.

The reported transfer experiment is particularly notable. The agent first received RL training in a mocked train-ticketing application and was then evaluated on the corresponding real-world counterpart using a physical phone. Task success increased from 28.8% to 48.1%. This does not show that simulation can fully replace real devices, but it does provide evidence that carefully designed mock environments can generate useful learning signals for real GUI interaction.

Why it matters—and what remains open

MAI-Gym connects several pieces that are often developed separately: environment execution, task synthesis, reward verification, benchmark evaluation, and online training. Releasing the framework, simulated apps, synthesis skills, verified tasks, and model checkpoint also gives researchers a basis for reproduction and controlled comparison.

The results should still be interpreted in the context of the reported benchmarks and environments. Mocked applications may differ from real software in state space, visual variation, and failure modes, while cross-app tasks may not cover the full diversity of real workflows. More testing is needed on unseen tasks, changing interfaces, and a broader range of physical devices. Even so, MAI-Gym presents a practical scaling strategy: use resettable environments to lower interaction costs, execution-based verifiers to improve reward reliability, and a shared pipeline to keep training and evaluation aligned.

Source: Hugging Face Daily Papers