Player-1 Teaches Game Agents Through Synthetic Experience

Introduction

Computer-use agents can already interpret screens and issue keyboard or mouse commands, but games expose the limits of simple screen-to-action imitation. Game interfaces are visually dense, their states change continuously, and many tasks depend on holding an input, steering precisely, or combining several actions over time. Player-1 proposes a way to move beyond static demonstrations: let an agent acquire additional control skills through scalable synthetic experience.

What the framework changes

Player-1 is a post-training pipeline built on computer-use models. Its design has three central components.

• A richer action space. The framework adds relative pointer motion, sustained inputs, and composite actions to ordinary keyboard and mouse commands. These capabilities are important for games because an action is often not a single click or key press. Movement may need to continue for a period, and several controls may need to be coordinated.
• Supervised initialization. Before reinforcement learning, the researchers fine-tune the base computer-use models on 6,044 curated gameplay trajectories. The trajectories pair task instructions with recorded actions and visual reasoning. This stage provides an initial grounding in observing a game and attempting the requested behavior, without claiming to cover every possible state.
• Synthetic reinforcement learning. Player-1-Gym contains 16 synthetic 2D and 3D environments organized around control demands shared by many games. Programmatically verifiable rewards make it possible to evaluate behavior automatically. Randomized observations and control dynamics provide variation, while PPO is used for multi-environment training.

Results

The reported results indicate that synthetic interaction can improve policies beyond their supervised starting point. On MineDojo-300, Player-1-9B and Player-1-35B-A3B improve over their respective SFT policies by 27.8 and 17.7 percentage points. Player-1-9B reaches 38.33% success, compared with 12.63% for the strongest evaluated baseline. The paper also reports gains over SFT on ViZDoom-10 and GameWorld.

The important point is not simply that one model posts a higher score. Games provide a compact test of a broader agent loop: perception must be connected to action, action must be sustained or composed correctly, and feedback must change later decisions. A synthetic gym makes that loop easier to scale than collecting every interaction from real games. Randomized observations and dynamics can also reduce dependence on a fixed interface or a single control pattern.

Why it matters

Player-1 presents a practical bridge between demonstration learning and trial-and-error learning for digital agents. It uses demonstrations to establish basic competence, then uses controllable environments to expose the policy to additional states and consequences. This recipe could be useful for studying general computer control where interaction data is expensive or difficult to verify.

There are also clear open questions. Synthetic environments may not capture the full variety of commercial games, and performance on benchmark tasks does not by itself establish robust transfer to open-ended, long-horizon play. The work therefore shows the value of synthetic experience, rather than proving that game control has become a solved problem.

Source: Hugging Face Daily Papers