Learning Foresight without Explicit Trajectories for 3D Diffusion Policies

Learning Foresight without Explicit Trajectories for 3D Diffusion Policies. By Zhongbo Zhang and five others, from the Dalian University of Technology. Published on the seventeenth of September. Its code repository is public and real: about twenty-five source files.

Three-dimensional diffusion policies are strong at generating geometrically grounded actions from what they currently observe. Successful manipulation needs more than knowing which motion is feasible now. It also needs a sense of where the interaction is heading. Existing policies mostly leave that foresight to emerge implicitly, as a by-product of learning actions. Nothing in their training asks them directly to anticipate, so whatever sense of direction they have is incidental.

Movement Trend Guidance provides the foresight directly, without introducing an explicit plan. The distinction matters. An explicit plan would spell out a trajectory for the robot to follow. This method gives the policy something softer: a sense of trend rather than a path. From a short history of observations, the policy learns a compact latent representation of how the interaction is evolving, and that representation, not a list of future positions, is what it carries forward.

During training, sparse future gripper states supervise that latent. At inference, only the latent is kept, serving as future-oriented conditioning next to the current observation. The future gripper states are never needed at run time. They shape what the latent learns, and then they are gone. It provides global conditioning for generating actions, and an additional gated FiLM branch is used only at the UNet bottleneck. The addition is small: three point five two percent more parameters than DP3, with the original formulation, dense actions and a receding horizon, left intact.

The method consistently improves on DP3 across RoboTwin2.0, LIBERO-40, and DexArt. In mixed training over fifty RoboTwin2.0 tasks, it reaches sixty-two point eight percent, against fifty-six point one. On LIBERO-40 it reaches seventy-one point nine three percent, against thirty-seven point zero eight. On five real-robot tasks, the figures are seventy-two percent and forty-nine percent. The largest jump is on LIBERO-40, where success roughly doubles, and the gain carries over from simulation to the real robot. Across all three settings the comparison is to the same base policy, DP3, so the difference is what the latent adds.

The future serves as a teacher rather than a script. Sparse glimpses of where the gripper will be shape the latent during training, and at run time the policy carries a sense of where the interaction is heading, without being told exactly where to move.