Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy
Tara Sadjadpour, Siming He, C. K. Wolfe, Haozhi Qi, Lea Wilken, S. Shankar Sastry, Claire Tomlin, Jitendra Malik
University of California, Berkeley
Published September 23, 2026 · on the feed September 28, 2026 · ▲ 13 · arXiv 2609.28660
The abstract
Human hand-object interactions (HOIs) provide a rich source of demonstrations for dexterous manipulation, but learning directly from them presents challenges in bridging morphology gaps, ensuring dynamical feasibility, and sim-to-real deployment. We present Morphometric Imitation, a three-stage framework that transforms reconstructed HOIs into zero-shot sim-to-real visuomotor policies. First, morphometric optimization (MMO) kinematically retargets human motion across hand morphologies while preserving demonstrated contacts. Second, residual reinforcement learning (RL) refines the kinematic reference using object pose and contact information from the human motion to produce dynamically feasible robot demonstrations. Third, these demonstrations are distilled into visuomotor policies. Across three robot hands and ten HOIs, MMO improves contact F1 over the strongest of five baselines by at least 8 points for every hand, while also improving the success rate of downstream dynamic retargeting by as much as 35 points. Ablations on the residual RL show complementary benefits from using object pose and contact information. Finally, the visuomotor policies achieve 89.3% zero-shot success in 300 real-world trials on 30 objects. Project page: https://morphometricimitation.github.io{this https URL}
The Brief
2:05A short podcast summary of the paper. Written and voiced by AI.
Transcript
Morphometric Imitation. From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy. By Tara Sadjadpour and seven others, from the University of California, Berkeley. Published on the twenty-third of September. Its code repository is a placeholder: a readme, and nothing to run yet.
Human hand-object interactions, recordings of people picking up and handling things, are a rich source of demonstrations for teaching robots dexterous manipulation. Learning from them directly runs into three obstacles. A robot hand is not shaped like a human one, so there is a morphology gap to bridge. Motion that looks right may not be dynamically feasible for the robot. And a policy learned in simulation still has to work in the real world.
Morphometric Imitation meets those three obstacles with three stages, one each. It takes reconstructed hand-object interactions as input and turns them into visuomotor policies that go from simulation to the real world zero-shot.
Stage one is morphometric optimization, or MMO. It kinematically retargets the human motion onto a robot hand of a different morphology, while preserving the contacts the human made. Stage two uses residual reinforcement learning to refine that kinematic reference, drawing on object pose and contact information from the human motion, and produces robot demonstrations that are dynamically feasible. Stage three distills those demonstrations into visuomotor policies.
The evaluation spans three robot hands and ten interactions. MMO improves contact F1 over the strongest of five baselines by at least eight points for every hand, and it lifts the success rate of the downstream dynamic retargeting by as much as thirty-five points. Ablations of the residual learning stage show that object pose and contact information each bring benefits the other does not. In the real world, the final visuomotor policies achieve eighty-nine point three percent zero-shot success over three hundred trials on thirty objects.
Each stage hands the next something more usable: contacts preserved across a change of hand, then motion a robot can physically carry out, then a policy that runs from vision on real hardware.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Humans are the best teachers of hands, but robot hands are not human hands. That is the wall.
You have recordings of people grabbing and handling everyday objects, and you want a robot hand to learn from them. But the robot's fingers differ in size and shape from a human's.
The road ahead
- 1 · The problem ~7 min Human recordings of handling objects are rich teaching material for robot hands, but a robot hand is shaped differently, so copying the motion breaks.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Reshape a human hand model to the robot's proportions, re-pose it to recover the human's contacts, then let residual RL make the motion physically work.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Human recordings become robot motion in stages: retarget kinematically, refine with residual RL, distil into a camera-driven policy, then deploy on a real robot.
- 6 · The experiment ~6 min You’ll guess: Trained from retargeted human hand-object interactions, what zero-shot success rate did the distilled visuomotor policies reach in real-world trials?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.