Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence

Pelican-Sim 1.0. A General World Model Simulator for Embodied Intelligence. By Shilong Zou and ten others, from the Beijing Innovation Center of Humanoid Robotics. Published on the tenth of September. Its code repository is a placeholder: a readme, and nothing to run yet.

Embodied intelligence needs a way to predict what comes next. A world model simulator takes visual context and robot actions, and predicts future observations that can support downstream learning and decision making. Making one general runs into several obstacles: robots differ in how their actions are represented, actions and pixels are different kinds of signal, dynamics vary widely, and rolling a video model forward takes many steps.

Pelican-Sim 1.0 is a general world model simulator for embodied intelligence, presented as a technical report and built around four design features that answer those obstacles one at a time. The goal is one model that can stand in for the real world across many robots and many tasks: given what a camera sees and which actions a robot takes, show what would happen next, well enough that policies can be trained and evaluated against it.

First, a unified action representation: a twenty-eight dimensional action value space that covers most mainstream embodiments, so that one model remains valid across heterogeneous devices. Second, action-visual injection. Action videos, rendered through URDF and through cameras, bridge actions and pixels, giving markedly better controllability across embodiments, scenes and tasks. Against alternative fusion baselines, that is a PSNR gain of zero point nine zero four.

Third, sparse mixture-of-experts layers add capacity for heterogeneous dynamics, and absorb the action modality while reducing conflict between modalities. That improves FVD by six point five three against the dense backbone. Fourth, efficient rollout generation. Causal adaptation and few-step distillation yield a four-step autoregressive simulator, five point six seven times faster than the thirty-five step model.

Trained on approximately one million real-world and simulated trajectories, the model shows large gains in action controllability and video quality. Over the strongest baselines evaluated, PSNR improves by four point six three six on AgiBotWorld Beta, two point zero eight on RoboMIND, and ten point three four three on RoboTwin. The adapted EWMBench DYN score rises by zero point four two six on RoboTwin.

The downstream tests ask whether the simulator is useful, not just accurate, and four applications on RoboTwin succeed. Adding five hundred generated trajectories to the fifty demonstrations per task raises policy success from seventy percent to ninety-three. Policy evaluation reaches a Pearson correlation of zero point nine nine four across five checkpoints. And relative success gains reach forty-seven point seven percent for action selection, and twenty point three percent for policy improvement. Put together, the simulator is used four ways: to make training data, to rank policies, to choose actions, and to improve a policy.

Qualitative tests across shifts in trajectory, scene, object, embodiment and viewpoint point to its potential as a general-purpose simulator. The design puts generality into the interface first. A shared action space and rendered action videos let one model address many robots, and the remaining features give it the capacity and the speed to be put to use.