DONIC Teaches Dexterous Robots to Coordinate Two Hands from Human Demonstrations

Introduction

For a robot to reproduce everyday human manipulation, controlling one dexterous hand is only part of the challenge. Many tasks require two hands to coordinate around the same object: one hand may stabilize it while the other turns, pulls, or reshapes it. Both hands must maintain contact, share force through the object, and adjust their roles as the object moves. Existing dexterous tracking systems have largely emphasized single-hand manipulation, while contact-rich bimanual data and learning methods remain limited.

DONIC, short for Generalist Dexterous Bimanual Tracking Control from Human Demonstrations, targets this gap. The method learns a single reference-conditioned policy for two Wuji dexterous hands across different operation categories and human-motion sources. Its goal is not merely to imitate one motion clip, but to generalize to new trajectories and objects without task-specific fine-tuning.

Key ideas

• BiHOI expands the data foundation. The accompanying multi-view dataset contains 1,575 sequences covering 229 objects and 14.77 hours of interaction. It focuses on sustained two-hand contact. Reconstructed human interactions are retargeted into robot reference trajectories, making the data usable for bimanual tracking.
• Rewards are organized around the object and each hand’s role. DONIC uses an object-centered per-hand reward. Object motion supplies shared task feedback, while each hand is supervised according to its contact role in the reference. This design addresses a central failure mode of bimanual control: two hands may individually follow their targets without actually coordinating through the object.
• Clip-Centered Advantages reduce reference bias. The method processes advantage estimates around each motion clip. This removes reference-dependent offsets while preserving differences within a trajectory, allowing one policy to learn from heterogeneous demonstrations without treating one reference source as intrinsically more rewarding than another.
• Successful rollouts become new demonstrations. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop. Executions that pass replay verification are recycled into the demonstration pool, so the policy can learn not only from the initial human data but also from behaviors it has already performed reliably.

Results and implications

In simulation, one DONIC checkpoint achieves 86.79% success on unseen trajectories involving training objects and 57.32% on held-out objects, without fine-tuning. The reported comparison shows gains over the authors’ stronger bimanual adaptations of DexTrack and Dexplore. Adding BiHOI improves all seven reported metrics on both held-out splits, raising success to 88.68% and 59.76%.

The contribution is therefore broader than a new dataset or a larger policy. DONIC connects three pieces of the generalization problem: sustained-contact data, feedback that represents cooperation at the object level, and a training loop that promotes verified successful behavior. Together, they offer a practical route from human motion capture to a reusable bimanual controller for contact-rich tasks such as opening, handling, and assembly.

The available material also sets an important boundary on the claim. The reported success rates are from simulation, and no broader conclusion about deployment on physical hardware is provided here. Real-world sensing noise, differences between robot hands, and unmodeled contact dynamics may still affect transfer. Nevertheless, DONIC makes a useful conceptual point: general bimanual dexterity requires more than placing two single-hand policies side by side. It requires a shared object-centered learning signal and an explicit mechanism for maintaining complementary contact roles.

Source: Hugging Face Daily Papers