Drive3R: Block-wise Streaming Geometric Foundation Models for Autonomous Driving

Drive3R. Block-wise Streaming Geometric Foundation Models for Autonomous Driving. By Tianyi Yang and six others, from Shanghai Jiao Tong University, Didi Chuxing, and one other institution. Published on the tenth of October. Its code repository is public and real: about twenty-five source files.

High-fidelity closed-loop simulation for autonomous driving increasingly relies on dense multi-view 3D reconstruction. Recent feed-forward geometric foundation models have revolutionized per-scene optimization, but applying them to continuous, long-horizon driving sequences is computationally intractable. GPU memory runs out, and global attention has quadratic complexity. The existing ways of scaling each break something. Causal streaming suffers from compounding scale drift and trajectory deviation. Rigid chunk-then-align heuristics fragment the global context and leave discontinuities in the trajectory. One approach loses the whole, the other loses the joins.

Drive3R is a block-wise streaming geometric foundation model, and it sits between two extremes. Fully bidirectional attention does not scale. Purely causal streaming scales, but loses local future context. Block-wise causal attention resolves the scalability bottleneck of the first while avoiding the loss that comes with the second.

Streaming can still drift over long ranges, so a second mechanism handles that. Confidence-aware re-anchoring detects when the streaming state has become unreliable. When it does, the model re-initializes its KV cache, and stitches the refreshed segment back onto what came before through an SE three alignment. Long-range drift is caught and reset rather than allowed to compound.

Extensive experiments report a new state of the art on nuScenes, KITTI, and Waymo. On nuScenes, the comparison is with the recent DVGT-2 baseline. AbsRel improves from zero point one eight zero six to zero point one zero eight one. Point-cloud Accuracy improves from zero point six seven five to zero point five three four four. And ATE improves from one point five eight one two to zero point nine zero nine two.

Drive3R pairs two decisions. Block-wise causal attention handles the length of a real drive without giving up the local future context that pure streaming discards. Confidence-aware re-anchoring then catches the drift that streaming still accumulates, by starting a fresh segment and aligning it back. Neither alone covers both failure modes the authors set out from: scale drift on one side, fragmented context on the other.