ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning

ThinkWorld. World Imagination Reinforces Chain-of-Dynamics Reasoning. By Chaehyun Kim and six others, from NVIDIA and KAIST. Published on the twenty-seventh of October.

Vision-language-action models have come a long way in robotic manipulation, yet how well they generalize is still capped by the diversity and coverage of robot demonstrations that come with action labels. Human videos hold far broader experience, of behaviors and of environments. Learning from them, though, runs into a basic trade-off. Data that scales well lacks explicit grounding in actions, and data with that grounding does not scale.

ThinkWorld is a learning framework for these models that grounds action reasoning in world dynamics, then strengthens that reasoning with reinforcement learning driven by world imagination. Each action thought is represented as a Chain-of-Dynamics, or CoD: a sequence of discrete motion tokens capturing the intermediate dynamics that connect the current world to a future state. A CoD also serves as a common action representation across embodiments.

Training starts by initializing CoD from demonstrations that do carry action labels. The framework then predicts the future that each CoD would bring about, and reinforces the ones whose predictions better match the outcome actually observed. That step is what opens the door to unlabeled video. A video without action annotations still shows a state changing into another state, and that observed transition becomes direct supervision. Human experience and robot experience can then refine the same reasoning space. Put plainly, the model is rewarded for motion plans that imagine the right future, and any video, labeled or not, can say what the right future was.

Experiments run both in simulation and on real robots. They show effective transfer from humans to robots, and strong generalization to tasks and environments the model has not seen. The abstract reports these outcomes in words rather than figures, so the claim to hold onto is the direction: experience gathered from human video carries over to robot behavior.

The framing turns imagination into a check. Rather than asking a video for the actions it never recorded, ThinkWorld proposes a chain of motion, imagines where it leads, and lets the video say whether that is where things went.