CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments

CrossBFM. Distilling a Shared Latent Behavior Space Across Humanoid Embodiments. By Tan-Dzung Do and eight others, from VinRobotics, the National University of Singapore, and four other institutions. Published on the twenty-ninth of September. No code repository is linked to it.

Behavior Foundation Models give a humanoid robot a policy that can be prompted through a latent behaviour space. One single vector in that space can stand for a motion to imitate, a pose to reach, or a reward to maximise. Forward-Backward representations are one way to produce such a space, and they work. But they cost hundreds of GPU-hours, and that is for a single robot.

Repeat the training for a second robot and the cost is not the only problem. The second run produces a second space, unrelated to the first. Each robot ends up with latents specific to its own body, which neither unify with the others nor transfer between them. A vector that means walk forward on one robot means nothing in particular on another.

CrossBFM changes what is treated as the valuable thing. Instead of each robot's policy, the latent space itself becomes the asset that transfers across embodiments. The opening comes from retargeting, which already supplies a frame-by-frame correspondence between how different bodies perform the same motion. With that correspondence available, one behaviour space can be distilled for several robots at once.

The distillation uses a unified encoder architecture with no robot-specific parameters, so the same encoder serves every training embodiment simultaneously. Distilling the behaviour space this way takes less than one GPU-hour. After the encoder, trackers conditioned on the latent turn it into whole-body control. They are trained with conventional PPO, in just ten more GPU-hours. The bill that used to run to hundreds of GPU-hours per robot now covers several robots at a small fraction of that.

On three distilled humanoids, all three prompting modes carry over. For motion tracking, the policy conditioned on the latent loses only zero point zero two five radians against its counterpart conditioned directly on joints. Goal reaching moves smoothly between poses, with no falls. And reward optimisation works for all forty-one reward prompts tested.

Two further experiments test how far the shared space stretches. Fitting the encoder on only a quarter of the motion corpus costs just five percent of tracking performance. Training the encoder on a subset of the robots and evaluating on one it has never seen recovers up to eighty-nine percent of the tracking performance achieved on seen robots. That is cross-embodiment generalisation, and the authors note it applies to robots of similar morphology. The whole pipeline was also verified on real robots, across all three prompting modes, and with latents produced by a flow-based generator.

The central reframing is to stop thinking of a behaviour model as something each robot owns. With the latent space shared, a single vector can prompt the same behaviour on several bodies, and the space is distilled in under a GPU-hour instead of being trained from scratch for each one. The stated limit belongs alongside the result: generalisation to an unseen robot was shown for robots morphologically similar to the ones it trained on.