ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

ReactHuman. A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs. By Yizhan Li and seven others, from the Université de Montréal, Mila, the Quebec AI Institute, and three other institutions. Published on the ninth of September. No code repository is linked to it, though it does publish its weights.

Reacting to a sudden physical hazard, like catching a slipping plate or dodging a falling knife, is a meaningful test of embodied intelligence. It is also a hard requirement if multimodal large language models are to serve as the decision core of household robots. Existing evaluations miss it in two ways. Some probe intuitive physics passively, through question answering over videos. Others target deliberate, long-horizon tasks such as navigation and rearrangement. None measures whether a model can turn physical understanding into immediate action when safety is at stake. Knowing that a plate is sliding off a counter is one skill. Deciding, in the moment, to reach for it and where to put your hand is another, and that second skill is the one a household robot would need.

ReactHuman is presented as the first physics-grounded benchmark for human-like reactive decision-making. The model under evaluation acts as the brain of a simulated humanoid facing sudden household hazards. Each reaction is scored by a five-metric suite along three axes: whether it is reasonable, whether it is safe, and whether it is physically grounded. The model is not asked what will happen. It is asked what to do, and then judged on what doing it achieved.

The benchmark spans seventeen event families and more than one thousand scenes, each reproducible bit for bit. Ground truth is exact and needs no annotation, because it is derived from rigid-body simulation running at two hundred and forty hertz. Some objects are adversarial, with an appearance that contradicts their physics: a foam anvil, a steel apple. Those objects separate a model that reads the motion in front of it from one that goes by what things usually look like. And every plan a model commits to is physically executed, so each decision has consequences that can be observed, rather than being graded on how plausible it sounds.

Seven representative multimodal models were evaluated with this harness, and reactive safety is far from solved. Models mishandle roughly one hazard in three. They act from fixed dispositions rather than from the scene in front of them. They trust appearance over motion, which is exactly the trap the foam anvil and the steel apple are built to spring. And even when the chosen action is correct, they miss interception points by distances on the scale of meters: the right idea, carried out in the wrong place. None of these failures shrinks as models get larger, so scale alone does not look like the way out.

Because its ground truth comes from simulation rather than from human labels, ReactHuman is offered both as a fine-grained diagnosis and as a scalable training signal, toward embodied agents that are physically grounded and aware of safety. The benchmark is released as a dataset on Hugging Face.