OmniCam: Omni-Camera Trajectory Generation via Geometry-Grounded Pose Token Learning

OmniCam. Omni-Camera Trajectory Generation via Geometry-Grounded Pose Token Learning. By Zhenyang Liu and seven others, from Fudan University, the Shanghai Innovation Institute, and two other institutions. Published on the tenth of October. No code repository is linked to it.

A camera trajectory decides how the viewpoint moves, in video generation, in scene reconstruction, and in robotic perception. Generating one from a sentence asks for two things at once. The path has to respect the geometry of the scene, and it has to frame the right target.

OmniCam is an autoregressive model that produces a sequence of camera poses from a single panorama plus a text description of the trajectory. What grounds it is geometry-grounded pose token learning, built from three parts.

First, a panoramic point-cloud encoder supplies omnidirectional geometric context. Second, poses are tokenized in a hybrid way: rotation as absolute, translation as relative, with quaternion signs kept temporally consistent. Third, geometric conditioning and semantic conditioning travel in separate streams, with an explicit three-dimensional anchor marking the target. Each part answers a piece of the opening problem. The encoder supplies scene geometry, the tokenization gives the pose sequence a consistent form, and the separate streams with their anchor keep track of what the camera should frame. The authors also build a dataset for the task, OmniCaT, holding two hundred and sixty-seven thousand seven hundred trajectories across four camera behaviors.

On the reported OmniCaT evaluation, against GenDoP retrained on the same data, OmniCam cuts trajectory errors by twenty-eight to forty-seven percent and the collision rate by sixty-five point eight percent. Measured against whichever baseline is best on each metric, the reductions are forty-three percent for ATE and sixty-two point three percent for collisions. Component ablations back the geometric and target-aware conditioning.

Fewer collisions is the number that speaks most directly to the opening problem, since a path that runs through a wall has ignored the scene. The downstream experiments carry the model into camera-controlled video generation and robotic active perception, the settings the work began by naming.