InstantHMR : Real-Time Human Mesh Recovery on Your Phone

InstantHMR. Real-Time Human Mesh Recovery on Your Phone. By Mohamed Adjel and two others, from NaturalPad and LAAS, CNRS. Published on the sixth of November.

Recovering a 3D human mesh from a single image, in real time, on a mobile device, has plenty of uses: robotics, biomechanics, AR and VR, and exergames among them. It is also hard. The current state-of-the-art models fall on one side of a trade or the other. Either they are too slow to run efficiently on a phone, or they give up accuracy to gain speed.

InstantHMR is a family of single-image regressors built for mobile inference. They are based on the Momentum Human Rig, the MHR parametric model, and they are trained only on publicly available annotations from the SAM 3D Body dataset. The aim is to stop choosing between the two sides: to be fast enough for a phone while keeping accuracy competitive.

The abstract gives the training ingredients rather than the internals, so the substance is in how the work is measured. To keep comparisons fair, the authors re-evaluate eleven state-of-the-art methods under one unified protocol, aligned with the closest MHR-based competitors. That protocol includes Meta's official MHR-to-SMPL conversion, and the reproduction results for those methods are reported. Latency is then measured on eight different targets, ranging from desktop TensorRT down to two phone NPUs.

On a Samsung Galaxy S23 NPU, InstantHMR recovers a body mesh of six thousand eight hundred and ninety vertices in five point five milliseconds per person. On an NVIDIA RTX 4070 GPU the same job takes zero point nine eight milliseconds. The authors report that this speed surpasses every existing method they evaluated.

There is one method that runs on mobile and slightly beats InstantHMR on accuracy, and on mobile InstantHMR is twelve times faster than it. Accuracy stays competitive, measured as PA-MPJPE: forty millimetres on 3DPW, forty-eight point eight millimetres on EMDB, and forty-one point seven millimetres on RICH.

The claim is a position on a trade-off curve rather than the top of a leaderboard. InstantHMR concedes a small margin of accuracy to the one mobile-compatible method ahead of it, and in return runs twelve times faster on mobile. That is the gap the work set out to close: mesh recovery quick enough for a phone, without accuracy falling away.