InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation

InterMimicGen. Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation. By Yucheng Zhang and ten others, from the University of Illinois Urbana-Champaign and NVIDIA. Published on the fifth of October. No code repository is linked to it.

Captured human-object interactions are rich supervision for humanoid loco-manipulation. They also come with three problems. They are sparse, they are heterogeneous, and robots cannot execute them directly. Rich as the signal is, there is not much of it, it does not arrive in one consistent form, and it does not run on a robot as captured. Any route from human recordings to robot skill has to deal with all three at once.

InterMimicGen is a self-evolving motion-imitation framework in which the robot motion data and a tracking policy improve each other. The data trains the tracker. The tracker, in turn, decides which new data survives to be used. Each side raises the other, and that loop is what the word self-evolving refers to.

It is built in three stages. The first consolidates motion-captured human-object interaction datasets and retargets them into humanoid robot references, preserving whole-body coordination and dexterous hand-object relationships. That answers heterogeneity: many sources become one collection of references for dexterous whole-body loco-manipulation, large and diverse. The second stage answers executability. It trains a physics-based generalist tracker that executes those references in simulation, on a humanoid with dexterous hands, at a scale and diversity the authors place beyond prior humanoid tracking systems for loco-manipulation.

The third stage answers sparsity, by closing a data flywheel. Each round makes small, task-preserving changes: to where an interaction takes place, and to how the body performs it. The tracker is fine-tuned on these variants, and only those whose simulated execution completes the task are kept, to seed the next round. With more iterations, the small edits compound into broader coverage around the sparse original demonstrations, while task semantics and motion quality are preserved.

The experiments report four results. Retargeting preserves contact across robot configurations. A single generalist policy tracks broadly. The set of executable motions keeps growing over augmentation rounds. And the behaviour transfers to real robots, beyond the simulation where the data was grown.

The framework is pitched as a path rather than a fixed dataset: from heterogeneous human demonstrations to a motion resource that keeps expanding. Simulation is the filter at every turn, so each expansion of the data is one the tracker has already shown it can execute, and the sparse originals become the seeds of a much wider neighbourhood.