OpenForgeRL Trains Harness-Native Agents Across Real Environments

Introduction

Modern AI agents increasingly depend on inference harnesses rather than a single model call. Systems such as Claude Code, Codex, and OpenClaw coordinate multi-turn reasoning, tool use, state tracking, and access to external environments. That makes them powerful in practice, but difficult to train with today’s open SFT and RL pipelines, which typically assume cleaner model-input/model-output loops.

OpenForgeRL targets this gap. Instead of asking researchers to rewrite a complex harness into a simplified training environment, it aims to train agents directly inside the same harnesses and environments in which they are deployed.

Key ideas

• Decoupling inference and training: OpenForgeRL inserts a lightweight proxy between the harness and the model. The proxy serves model calls made by the harness while recording them as training data that can be consumed by a standard RL codebase such as veRL.
• Containerized rollouts at scale: A Kubernetes orchestrator runs each rollout in its own remote container. This is important for agent tasks that involve tools, browsers, GUI systems, or other external resources that need isolation and reproducibility.
• Harness-native training: The framework is designed to work with real, stateful, multi-process harnesses rather than forcing them into a simplified abstraction. This allows training to reflect the actual behavior of deployed agents.
• Broad validation settings: The paper evaluates both tool/Claw-style agents and multimodal GUI, browser, and computer-use agents. Reported results include OpenForgeClaw reaching 31.7 pass^3 and 55.9 pass@3 on ClawEval, plus 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager.

Why it matters

The main contribution is infrastructure. Agent quality is not determined only by the base model; it is also shaped by the harness, tool-selection policy, verification behavior, and interaction with the environment. If those components are excluded from training, researchers may optimize a proxy problem rather than the real agent workflow.

OpenForgeRL makes it easier to study that full loop. The authors report that harness choice has a major effect on learnability, and that RL can improve reliability-related behaviors such as self-verification, tool coverage, and completion of multi-step plans. At the same time, they note that important capabilities such as error recovery remain weak.

For the open-source agent ecosystem, the framework points toward a more practical path from prompted agents to trained agents. If adopted broadly, it could help researchers compare harness designs, RL algorithms, and environment setups without rebuilding the entire inference stack for every experiment.

Source: Hugging Face Daily Papers