GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

GeoState. Learning View-Decoupled Geometry States for 3D World Modeling. By Hongchang Liu and seven others, from Tongji University, the Hong Kong Polytechnic University, and two other institutions. Published on the fifteenth of October. Its code repository is worked out: about forty source files, with a readme and a licence.

Recent 3D world models increasingly use features from geometry foundation models as their predictive world states, drawing on the rich geometric priors those features carry. The catch is that the features depend on the view. Scene dynamics get entangled with variation caused by the camera, and that makes the evolution of the state harder to predict. A change in the features might mean the scene moved, or it might mean only that the camera did, and a model predicting the next state has to untangle the two before it can predict either.

GeoState is a geometry representation decoupled from the view, built for predictive 3D world modeling. It separates how the scene state evolves from how a camera happens to observe it. The scene gets a state of its own, and the camera becomes a separate matter of how that state is looked at.

A scene tokenizer aggregates features from multiple views into a single view-decoupled scene state, and a geometry decoder queries that state using camera rays. An autoregressive world model predicts future scene states, while camera motion is modeled separately. Because of that split, a predicted state can be decoded under predicted cameras or under cameras specified from outside. The rollout also keeps a single scale, derived from the context, from start to finish, with no scale alignment frame by frame.

In experiments, GeoState outperforms prior 3D world models on Waymo. It supports cross-view querying that stays geometrically consistent, and it generalises zero-shot to KITTI and nuScenes. The cross-view result is the one the design most directly predicts: if the state really is independent of the view, querying it from another camera should agree with the first.

The move is a separation. What the scene is doing and where the camera is pointed are modeled apart, so the world model has to predict only the first, and the second can be chosen afterwards.