CaPtain Turns a Small Open VLM into a Code-Driven Robot Agent

Introduction

Robot agents are increasingly being framed as software agents that can reason, call tools, and revise their plans. Code-as-policy (CaP) applies this idea to manipulation: instead of predicting only low-level actions, a vision-language model writes programs against robot APIs, observes the returned results, and decides what to do next. The approach offers a flexible interface, but it creates a difficult learning problem. A request may move an object in the wrong direction, while the consequence becomes visible only several turns later. The training signal, meanwhile, is often just an end-of-episode success or failure.

The CaPtain pipeline

CaPtain is an open post-training pipeline designed to make this setup work with a small open VLM. Training and deployment take place inside CaPtain-Harness, which exposes low-level perception, geometric reasoning, and guarded-motion primitives across simulation and real hardware. The agent must combine these interfaces into executable code, inspect the results, and continue the interaction over multiple turns.

The pipeline has two stages. First, supervised fine-tuning uses synthetic trajectories generated with GPT-6 Astra. This phase teaches the base model how to use the interface, structure a manipulation program, and interact with the environment. The second stage applies multi-turn reinforcement learning, allowing the agent to improve its decisions through task outcomes.

The central technical component is a physical critic. During simulation training, the critic can read privileged physical state that is never exposed to the policy. It estimates how much each request changes the predicted probability of eventual success, then uses that estimate to distribute credit across a long sequence of interactions. This is intended to address delayed effects without giving the policy information unavailable at deployment.

Reported results

Starting from Qwen3.5-4B, which has a reported zero-shot success rate of 0%, CaPtain-4B reaches 55.4% on the held-out LIBERO split after supervised fine-tuning and 73.8% after reinforcement learning. On LIBERO-PRO, it reaches 78.5% and surpasses several frontier-model agents. The reinforcement-learning improvement also remains visible under three perturbation suites, with gains ranging from 17.6 to 21.2 percentage points.

Without target-domain training, the system reports 83.3% on SimplerEnv WidowX, exceeds frontier models on CaP-Bench, and can run on a real robot after simulation-only adaptation. These results suggest that a small open model can become a useful code-driven manipulation agent when the interface and post-training procedure are designed together.

The work also clarifies its limits. The policy does not receive privileged state at test time, but the reported training process depends on a particular harness, simulator, and task distribution. Broader real-world generalization therefore remains an open question. The release of code, data, and weights should nevertheless make it easier to reproduce the pipeline and investigate how far code-based robot control can scale.

Source: Hugging Face Daily Papers