ODE Policy: Robot Learning via Action Dynamics, Not Actions

ODE Policy. Robot Learning via Action Dynamics, Not Actions. By Jindou Jia and eleven others, from Nanyang Technological University. Published on the first of November.

Most robot policies, whether they are trained by imitation learning or by reinforcement learning, do the same basic thing. They map an observation to action values at discrete control steps. The continuity of physical motion, the fact that a real arm moves smoothly from one position to the next, is left for the policy to learn implicitly.

ODE policy changes what is being learned. Instead of the action itself, it learns the dynamics of the action: how the action should change. A neural ordinary differential equation learns the vector field of the action, and at deployment each new action is integrated forward from the previous one. Because every action grows out of the last, the executed trajectory is continuous by construction rather than by training. A conventional policy outputs where the action should be at each step. This one outputs which way the action should move from where it already is, and the integration turns that into the next action.

Training matches the integrated actions to their targets. In imitation learning this is done over short chunks of actions, and in reinforcement learning over single control steps. Behaviour can also be multi-modal, with more than one valid way to act. To capture that, a lightweight diversity head randomizes the direction of the field.

The evaluation is broad. ODE policy is tested as a visuomotor policy, as the action head of vision-language-action models and of world action models, and as the actor in model-based reinforcement learning, against fourteen baselines. The tasks span unimanual, bimanual, dexterous, locomotion, and loco-manipulation settings. That is one formulation slotted into several quite different roles, from a standalone policy to a component inside larger models, across both of the training regimes it was designed for.

In imitation learning, compared with flow matching after a single epoch, it reaches five point nine times the success, five point four times the generalization, and three point nine times the mode balance. It also generates actions ten point eight times faster. Its motion is, for the most part, smoother than the human demonstrations it learned from. Those first figures come after only one epoch of training, which is where the claim of efficiency in learning shows most plainly.

On fourteen real-world tasks, across single-arm, dual-arm, dexterous, and humanoid robots, it achieves the highest success rate. In reinforcement learning, it speeds up SuperMimic for humanoid motion tracking and smooths the actions it produces.

The authors describe the approach as efficient in learning, in data, in parameters, and in execution, and present it as a promising path to scaling robot learning. The shift underneath is easy to state. Policies have left continuous motion to be learned implicitly. Learning how the action changes, and integrating it from one step to the next, makes continuity a property of the policy's structure instead of something it has to pick up along the way.