HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
Kyochul Jang, Seohyeon Park, Ohchul Kwon, Sangjun Park, Junhyeok Choi, Seungyeop Yi, Chaeyun Kim, Sangkyu Lee and 4 others
Seoul National University · University of Massachusetts Amherst · Google Research
Published October 1, 2026 · on the feed October 15, 2026 · ▲ 28 · arXiv 2610.02089
The abstract
As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable tool and coordinating manipulation and, when needed, locomotion to complete the task. Existing benchmarks do not jointly evaluate these capabilities on a humanoid. We introduce HumanoidToolBench, an 18-task benchmark spanning three scenarios, three execution levels, and two tool-set modes, together with ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1. Evaluation of seven policies in simulation and three on the real robot reveals substantial gaps between selecting a suitable tool and completing the task. Focused GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued task execution under unrelated instructions. Code and data are available at https://snu-pi.github.io/HumanoidToolBench/.
The Brief
1:41A short podcast summary of the paper. Written and voiced by AI.
Transcript
HumanoidToolBench. Benchmarking Humanoid Tool Use from Selection to Mobile Execution. By Kyochul Jang and eleven others, from Seoul National University, the University of Massachusetts Amherst, and one other institution. Published on the first of October. Its code repository is worked out: about one hundred source files, with a readme and a licence.
As robot hardware and learning methods improve, humanoids need tools to do things beyond their own physical limits. Using a tool well is more than one skill. The robot has to pick a suitable tool, then coordinate manipulation and, when the task calls for it, locomotion to finish the job. Existing benchmarks do not evaluate these capabilities together on a humanoid.
HumanoidToolBench asks both halves of the question. Can a policy select the right tool, and can it then carry the task through to completion, moving around if it has to.
The benchmark has eighteen tasks. They span three scenarios, three execution levels, and two modes for the tool set. Alongside it comes ToolBook, a dataset of three point one thousand demonstrations, collected both in simulation and on a real Unitree G1 humanoid.
Seven policies were evaluated in simulation, and three on the real robot. The results show substantial gaps between choosing a suitable tool and actually completing the task. Focused probes of GR00T N1.7 add two observations. Its selection accuracy drops on tools it has not seen. And it keeps executing the task even when given unrelated instructions.
Picking up the right tool turns out to be only the start. The distance from a correct choice to a finished task is the thing this benchmark makes visible, and its code and data are public.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free