GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0. Pretraining and Scaling a World-Action Model for Robotic Manipulation. By AgiBot Research Team and forty-four others, from AgiBot. Published on the fourth of September. No code repository is linked to it.

A world-action model predicts future states and uses them to guide what a robot does, which lets it learn both from video with no action labels and from interaction data where actions are recorded. Most such models inherit a pretrained video generator as their starting point. As a result, how to pretrain a world-action model itself, and how it scales, has been left underexplored. The open question is what happens when the whole model is trained for manipulation from the start, and how far more data takes it.

Genie Envisioner Act 2.0, or GE-Act 2.0, takes the other route: every trainable generative and action component is initialised from scratch on manipulation data. It combines three parts. A control-oriented autoencoder, called CoAE. A single-step visual planner, called SVP. And an inverse dynamics model, called IDM.

CoAE keeps the information relevant to actions and instructions even under aggressive compression. SVP produces a complete future state in one differentiable pass. Because of that, visual planning and inverse dynamics can be pretrained separately, each on data that complements the other. The components are then trained jointly with knowledge-aligned selective optimization, or KASO. KASO reduces mismatched supervision by keeping only those predicted futures judged behaviourally compatible with the action that was actually recorded. A predicted future that does not fit the recorded action would teach the wrong lesson, so it is left out of the supervision.

Pretrained checkpoints are evaluated directly, with no per-task fine-tuning, on one hundred tasks across twenty manipulation skill groups, with held-out scenes, backgrounds, lighting and object instances. Scaling co-training data from three hundred hours to thirty thousand hours raises success on G1-OP from seventeen point one percent to forty-four point one percent.

On G2-90D, success rises from thirteen point four percent to thirty-one point one percent. That embodiment makes up less than two percent of the co-training data, yet it improves by seventeen point seven points, which the authors read as a sign of cross-embodiment transfer. Gains appear in nineteen of twenty skill groups on one embodiment and eighteen of twenty on the other.

Coverage of a specific skill strongly tracks zero-shot out-of-distribution success, with a Pearson correlation of zero point eight and a Spearman correlation of zero point eight five. Under the same protocol, the model grounds references to object, colour, shape and position in at least ninety percent of trials. It also follows explicit instructions when they conflict with a behaviour it had already committed to, or with a conventional association in the scene. In other words, the instruction wins over habit, whether that habit is the model's own ongoing behaviour or what a scene usually suggests.

The through line is that a world-action model can be built and scaled on its own terms. Nothing is borrowed from a video generator, the planner and inverse dynamics can be pretrained on separate data, and KASO filters what the joint training learns from. More data then lifts success broadly, including on an embodiment that barely appears in the mix.