Reflex: Visually Grounded Reactive Humanoid Control

Reflex. Visually Grounded Reactive Humanoid Control. By Taoyang Jia and eight others, from the University of Washington, Stanford University, and four other institutions. Published on the eighth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

People react to the world without thinking about it. They return a tennis shot, catch a cup as it falls, step aside for a passing cart, or take a box tossed to them while unloading a truck. Each of these means anticipating a moving target and answering with the whole body, inside roughly a second. For a humanoid robot this kind of visual, reactive, whole-body control is still hard, because three things have to come together. The whole body has to be coordinated. The robot has to respond to how an object is moving, in real time. And perception has to be grounded in egocentric vision, what the robot sees from its own viewpoint.

The authors choose catching thrown boxes as a task that demands all three at once. Existing systems cover only part of it. They rely on end-effectors, on motion capture, or on objects that have been marked to be easy to track.

Reflex is a three-stage framework that learns these capabilities one stage at a time. What links the stages is a compact representation of object dynamics, a summary of how the box is moving that each stage can share. The bet is that splitting the problem this way gives each capability its own direct training signal, and makes the visual part substantially more scalable to learn.

In the first stage, a state-based policy learns whole-body catching through reinforcement learning. In the second, the policy learns to infer the dynamics of a moving box from observations that are delayed and incomplete, which is closer to what a real robot gets. In the third, an egocentric RGB-D encoder learns to recover that same dynamics representation directly from what the camera sees. During this last stage the whole-body controller is kept fixed, so vision only has to learn to produce the representation the controller already understands.

In simulation, Reflex catches the box eighty-five point seven percent of the time from egocentric RGB-D. Given privileged observations of the box instead, the success rate is eighty-seven point nine percent, so seeing through its own camera costs very little. Temporal visual history turns out to matter a great deal. Without it, success drops by thirty-five point six points on centered throws, and by thirty-nine point eight points on throws offset to the side.

On a real Unitree G1, the robot catches boxes thrown by people. From two and a half metres it succeeds sixty-five percent of the time. From five metres it succeeds forty percent of the time. Measured against human catchers at the same distances, that is seventy-six percent of their success at the shorter range, and fifty-nine percent at the longer one.

The structure of the solution mirrors the structure of the difficulty. Three capabilities were hard to learn together, so Reflex learns them in sequence and connects them through one small description of how the object moves. Vision is taught last, against a controller that no longer changes, and that ordering is what lets a humanoid catch a box thrown by a person from what its own camera sees.