AgenticRoboRL Brings Post-Training to Robot Planners

Introduction

Many agentic robotics systems separate decision-making from execution. A language model plans the task and selects the next operation, while a robot harness exposes skills, APIs, and a control loop. Recent progress has often come from improving the harness or adding external memory. The planner itself, however, is frequently treated as a fixed, closed-source model. AgenticRoboRL explores a different idea: post-train the planner inside the robot system, much as coding agents are improved through supervised fine-tuning and reinforcement learning.

How the framework works

AgenticRoboRL does not replace the robot controller. Instead, it trains the language-model planner through the existing harness. A harness may let the planner invoke a VLA policy as a robot skill, or generate Python code that operates through robot APIs. The execution components and control loop remain fixed; the trainable part is the planner.

The recipe has two stages:

• Supervised fine-tuning: Successful trajectories from a stronger teacher are converted into the requests the student would actually receive at each turn. This teaches useful task decomposition and tool-use patterns before reinforcement learning begins.
• Multi-turn reinforcement learning: The model performs complete episodes inside the harness and simulator. The simulator’s task-success verdict serves as the reward. Because manipulation often requires repeated planning, tool calls, and corrections, the method optimizes a sequence of interactions rather than an isolated response.
• Harness-specific adapters: Robot harnesses can expose very different execution mechanisms. Adapters record planner calls, token IDs, and log probabilities, then report episode grouping and outcomes to a shared training pipeline. This allows the RL infrastructure to remain common while the execution systems stay unchanged.

The released implementation connects two types of harness: one that invokes a VLA policy and another that generates control code. The abstraction is important because it separates the question of how a robot executes actions from the question of how its planner decides what to do.

Results and implications

On LIBERO-PRO, the 9B planner’s success rate rises from 18.1% to 71.0% without external memory. That is close to the 72.1% reported for Codex (GPT-5.5) with external memory. The comparison suggests that large gains do not necessarily require a larger planner or a larger memory system; learning from execution outcomes can substantially improve how an existing model uses its tools.

The engineering implication is equally notable. Developers can retain an existing VLA policy, robot API, and control loop, then add an adapter that connects interaction logs with episode-level outcomes. This offers a more modular path toward reinforcement learning across robot systems and makes it easier to study planner quality independently of the underlying controller.

The evidence should still be interpreted with care. The reported benchmark is LIBERO-PRO, and it remains open how well the learned behavior transfers to other harnesses, physical robots, and longer-horizon tasks. A planner may also become specialized to the execution interface used during training. Even so, AgenticRoboRL presents a clear shift in emphasis: embodied intelligence can improve not only by upgrading the robot’s tools, but also by training the model that coordinates them.

Source: Hugging Face Daily Papers