Agent Lightning v1.0 Brings Reinforcement Learning Into Agent Harnesses

Introduction

An agent is rarely just a language model. Search, coding, and tool-using systems depend on a harness that manages context, executes tools, and determines how the interaction loop proceeds. Agent Lightning v1.0 examines how reinforcement learning can be connected to these real deployment-time systems instead of forcing them into a training-specific environment.

Key points

• The harness becomes part of post-training. The paper calls this approach “harnessed agentic RL.” In conventional agentic RL, the training engine controls the environment loop. In the new setup, the harness owns that loop, while the trainer receives only a stream of LLM request-response pairs.
• A proxy connects arbitrary agents to training. Agent Lightning uses an LLM endpoint proxy to connect existing agent harnesses with RL infrastructure. This disaggregated design aims to reduce the amount of application logic that must be rewritten for training.
• Data plumbing is a research problem. A sequence of model calls is not automatically a clean RL sample. The system must decide how to retokenize requests, merge calls, calculate advantages, normalize losses, and schedule work across the training backend. Poor choices in any of these areas can make training unstable or ineffective.
• The framework is intentionally small. Agent Lightning v1.0 is implemented in approximately 3,500 lines of code. It supports instruction-following, search, and coding agents, and includes data-cleaning and training workflows for coding-agent RL.

Why it matters

The most concrete result comes from the coding-agent experiment. Using 6K training examples and modest compute, reinforcement learning raised Qwen3.5-9B’s SWE-bench Verified score from 41.8% to 56.4%, an absolute gain of 14.6 points. The result suggests that agentic RL does not necessarily require a massive, highly specialized training stack to produce meaningful improvements in software-engineering tasks.

The broader contribution is conceptual as much as practical. Agent Lightning treats the harness—not merely the base model or the trainer—as a first-class component of the learning system. That framing could make it easier to study how multi-turn tool use, changing context, and control-flow decisions affect credit assignment. It also highlights an important limitation of current abstractions: logging model calls is simpler than representing the full trajectory an agent experiences.

The material does not establish that the framework will generalize equally across every model, task, or harness. Its value at this stage is providing a compact testbed and a reproducible workflow for investigating the engineering choices that determine whether harness-based agentic RL actually works.

Source: Hugging Face Daily Papers