DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations
Mingze Sun, Ziang Cao, Yuan Fei, Qiankuan Tao, Zihan Wang, Tianqi Liu, Junhao Chen, Haiwen Diao and 3 others
Ropedia
Published October 21, 2026 · on the feed October 22, 2026 · ▲ 161 · arXiv 2610.29997
The abstract
Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.
The Brief
2:25A short podcast summary of the paper. Written and voiced by AI.
Transcript
DONIC. Generalist Dexterous Bimanual Tracking Control from Human Demonstrations. By Mingze Sun and ten others, from Ropedia. Published on the twenty-first of October. Its code repository is worked out: about seven hundred source files, with a readme.
Dexterous reference tracking has made recent progress, but existing models focus mostly on a single hand and a narrow range of interactions. The two-handed behaviours common in everyday life are left unaddressed. Two things make a generalist bimanual tracker particularly hard to build. Contact-rich bimanual manipulation data is scarce. And coordinating contacts between two hands is more complex than managing one, because each hand's contacts now have to fit with the other's.
The paper answers with two contributions, one for each difficulty. DONIC is presented as the first generalist bimanual tracker, trained jointly across operation categories and across sources of human motion. Alongside it comes BiHOI, a multi-view dataset that emphasizes sustained two-handed interaction. Its reconstructed interactions are retargeted into robot references, so human demonstrations become targets a robot can track.
Training combines proximal policy optimization with behavioural cloning, in a feedback loop. Executions verified as successful are recycled into new training demonstrations, so the policy's own successes feed back into what it learns from. An object-centered per-hand reward keeps feedback on how the object moves, while supervising each hand according to the contact role its reference gives it.
A third piece, Clip-Centered Advantages, removes offsets in the advantage that depend on the reference, while keeping the differences within a trajectory. Together these designs let one single policy learn coordinated manipulation across references that are very different from each other.
BiHOI holds one thousand five hundred and seventy-five sequences spanning two hundred and twenty-nine objects. Without finetuning, a single checkpoint reaches eighty-six point seven nine percent success on unseen trajectories of training objects.
On held-out objects it reaches fifty-seven point three two percent. Those results surpass the state-of-the-art bimanual tracking baselines by seventy-seven point three six percentage points, and by forty-five point seven three points, respectively.
One policy, one checkpoint, two hands. The generality comes from two places at once: a dataset built around sustained two-handed interaction, and a training recipe whose rewards know which hand is meant to be doing what.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Two hands are harder than one. Here is why one policy struggles.
You want one robot policy that drives two hands to copy many human demonstrations: one hand holds a jar while the other twists the lid.
The road ahead
- 1 · The problem ~7 min One shared policy must drive two robot hands through many different human clips, but scarce data, shifting hand roles and uneven clip difficulty keep derailing it.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min DONIC trains one shared policy with reinforcement learning plus imitation of recycled successes, an object-centered per-hand reward, and per-clip advantage centering.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Human clips become robot references, PPO and BC train one policy, verified successes are recycled, and one checkpoint then tracks new clips without finetuning.
- 6 · The experiment ~6 min You’ll guess: Without finetuning, what success rate did the single checkpoint reach on unseen trajectories of training objects?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.