RoboCoach: World Models as Active Coaches for Compositional Robot Skills

RoboCoach. World Models as Active Coaches for Compositional Robot Skills. By Jiajun Liu and nine others, from Renmin University of China, Tsinghua University, and three other institutions. Published on the thirtieth of September. Its code repository is worked out: about eighty-five source files, with a readme and a licence.

Long-horizon manipulation reuses the same robot skills across many different compositions of a task. Improving those compositions by collecting more end-to-end demonstrations is expensive. A system that improves itself in a practical way has to make two decisions: what to teach next, and where that supervision should be applied. Collecting more demonstrations of the whole task answers neither question. It spends effort evenly, whether or not a given skill is the one holding the composition back.

RoboCoach lets a world model make both decisions. It is a coaching framework in which failures are imagined first, and those imagined failures guide which demonstrations to request and which experts to update. Rather than spending real demonstrations everywhere, the robot is coached on the subtasks where the world model sees it falling short.

The engine is a loop called Route-Imagine-Diagnose-Improve, or RIDI. Reusable skill experts are executed inside CoachWorld, the authors' shared world model conditioned on actions. A progress judge watches each imagined run and records the first subtask that fails to complete. Those records are then aggregated, and the aggregate decides two things: which subtask demonstrations to acquire, and which expert adapters to update. The diagnosis is local by design. Because the judge marks the first subtask to fail, blame lands on one skill in one composition, and the fix can be aimed at that skill's expert rather than at the whole policy.

Imagination has to track reality for this to work, and the authors check it. Across two simulation suites and two real-robot platforms, imagined and deployed success correlate over twenty-two pairs of task and policy, with a rho of zero point eight four. In controlled comparisons with matched data budgets and matched update schedules, the coaching method beats the baselines.

With only one hundred and fifty additional subtask demonstrations, success on Franka climbs from thirteen point three percent to seventy-five percent. On AgileX it climbs from forty percent to eighty-three point eight percent. The coached experts also carry over to four held-out compositions, averaging thirty-five percent success. A shared-policy baseline, updated with demonstrations acquired uniformly, scores zero on those same compositions.

In this framework the world model works as an active coach, almost a diagnostic instrument. It points at the first place a composition breaks, and that pointer decides where scarce human demonstrations go. On the authors' reading, this is how imagined failures become targeted supervision for improving a modular policy.