Fingers as Legs: Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand

Fingers as Legs. Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand. By Amirhossein Kazemipour and two others, from ETH Zurich. Published on the fifteenth of September. No code repository is linked to it.

A robotic hand that walks has to do everything with the same fingers. They must move its body, carry its weight, and interact with the world around it. The question is whether an anthropomorphic hand can learn all of that while keeping its existing finger design and position controller, rather than being rebuilt for walking. Walking and manipulating usually belong to different parts of a robot. Here they have to share one set of fingers.

The authors show that it can, using reinforcement learning. The platform is self-contained, with onboard power and computation. The learning approach takes into account that the hand's fingers are unequal, and training happens in a simulator calibrated from measurements of the real hardware. The simulator, in other words, is tuned to this particular hand rather than to an idealized one.

In simulation, the hand moves faster using the authors' reward formulation than using tuned rewards originally designed for four-legged robots. On the real hardware, separate policies trained for each task make the hand crawl, steer, and recover from falls, all without a tether.

Locomotion is only half of the demonstration. While holding up its own weight, the hand also types successive keyboard commands without vision, and pushes an object to targets using visual feedback from overhead. Both tasks happen while the hand is standing on its fingers, so the fingers doing the work are also the ones keeping it up.

What the hand becomes is a compact mobile manipulator that needs no separate way of moving. The fingers are the legs, and the same fingers, a moment later, are the hand again.