Robots Learn from Robots: A 96-Unit Test of Collective Experience Learning

Introduction

For embodied AI to move from demonstrations to large-scale deployment, model capability is only part of the challenge. Robots also need a continuing supply of real-world experience. Human operators can demonstrate actions and correct mistakes, but their availability is unlikely to grow as quickly as the number of deployed robots. A team from Tsinghua University, Infinigence AI and AgiBot has proposed Collective Experience Learning, or CEL, as a way for robots to learn not only from people but also from one another.

What the study reports

• Experience is shared across the fleet. A strategy that works for one robot can be stored and retrieved by others facing a similar situation. Successful trials provide examples to imitate, while failures help identify where a policy needs improvement.
• Five real-world tasks were tested. The evaluation covered flower arranging, folding clothes, charging a phone, assembling a stool and packaging a phone.
• Performance was high on held-out tests. Two robots that had not participated in training completed 600 trials with 15 failures, for an overall success rate of 97.5%. The reported success rate for each task was above the average of six human operators.
• A multistep industrial task improved substantially. Phone packaging rose from 79.2% to 98.3% after collective experience learning; only two of 120 consecutive trials failed.

The infrastructure behind the method

The system was built on RLinf and used a fault-tolerant asynchronous training architecture. Robots could upload experience as soon as they completed an episode instead of waiting for the entire fleet to finish. The design also aimed to keep training running despite network instability, device disconnections or hardware failures.

In the 96-robot test, the cluster collected 9,834 execution rounds. Twenty-two robots experienced 30 failures, yet the resulting loss of robot operating time was reported as 1.3% of the total. This matters because real-world reinforcement learning systems must tolerate imperfect hardware and uneven task completion rather than assume a clean laboratory setup.

At the algorithmic level, the team introduced Failure-guided Experience Retrieval fine-tuning, or FERT. A long manipulation episode may contain thousands of actions while receiving only a final success-or-failure label. FERT therefore uses successful trajectories to retrieve useful behaviors and failed trajectories to locate potential corrections. The goal is not simply to accumulate more data, but to make sparse feedback more actionable.

Why scale matters—and what remains open

The reported results show a positive scale trend: increasing the training fleet from 16 to 96 robots raised success from 86.7% to 98.3%. This suggests that deployment scale could become a source of learning rather than merely an operational cost. More robots can explore in parallel, and their experience can be recycled across the group.

The evidence is still limited to five tasks, so broader transfer across objects, environments and robot embodiments remains to be demonstrated. Experience quality, cross-platform differences, data filtering and safety constraints will also determine whether robot-to-robot learning can be used reliably in production. Even so, CEL offers a concrete route from single-robot improvement to continual learning at the fleet level.

Source: QbitAI