Robots Learn as a Fleet by Sharing Deployment Experience

Introduction

A robot policy that works in a controlled experiment may still struggle with the small variations found during deployment. A common way to improve it is to have a teleoperator watch the robot and intervene when it fails. That workflow becomes difficult to sustain as the number of deployed robots grows: monitoring every machine and supplying corrective demonstrations creates a human bottleneck.

The paper Learning from Fleet Experience: A Collective Learning Paradigm for Robots proposes Collective Experience Learning, or CEL. The idea is to let a fleet improve a shared policy through the experience already generated during operation, rather than relying on a new round of expert demonstrations for every failure.

How the approach works

• Failures guide the search for useful experience. CEL is implemented through Failure-guided Experience Retrieval Fine-tuning, or FERT. When a robot encounters an unsuccessful execution, the system uses the surrounding failure context to search a shared database of successful trajectories.
• Successful actions become supervision. Retrieved trajectory segments provide action sequences associated with similar situations. They can therefore guide policy updates toward behaviors that worked elsewhere in the fleet. A failure is treated not only as an error to record, but also as a query for relevant positive experience.
• Less dependence on intervention during deployment. FERT is designed to improve the policy without collecting additional expert demonstrations or corrective actions. This does not remove the need for an initial policy and data; rather, it targets the costly human feedback loop that normally follows deployment failures.
• Asynchronous coordination keeps learning running. The Fault-tolerant Asynchronous Training System, or FATS, coordinates experience collection, policy optimization, and policy distribution. Local interruptions at individual robots do not have to stop the entire fleet-level process.

Reported results

The study evaluates CEL on five real-world manipulation tasks. The resulting policies reach success rates between 93.3% and 99.2% on robots excluded from training data collection, with 120 trials per task. On a multi-stage phone-packaging task, the initial success rate is 79.2%; training with 96 robots raises it to 98.3%. When the collection time per robot is held constant, larger fleets generally contribute more experience, and FERT usually achieves higher success rates across the tested fleet sizes.

Why it matters

The central shift is from improving robots one at a time to treating the fleet as a shared learning system. More robots create more opportunities to encounter variations in the environment, while a shared experience store allows a useful behavior discovered by one machine to benefit others.

The available material mainly reports the framework and aggregate experimental outcomes. It does not provide full details on retrieval baselines, data-quality filtering, or performance under broader task and hardware shifts. Whether the approach remains reliable with heterogeneous robots and more severe distribution changes therefore requires further study. Still, CEL points to a practical deployment loop in which robot fleets can turn their own successful executions into policy improvements, reducing the need for continuous expert correction.

Source: Hugging Face Daily Papers