Towards Real-Time Full-Waveform LiDAR Transformers via Intensity-Guided Token Reduction and Physics-Aware Augmentation

Towards Real-Time Full-Waveform LiDAR Transformers via Intensity-Guided Token Reduction and Physics-Aware Augmentation. By Kotaro Oishi and five others, from Keio University. Published on the fifth of November.

LiDAR is a critical sensor for autonomous driving and for robotics, but it has well-known weak spots: fog, rain, transparent objects, and surfaces that reflect very little light. Full-waveform LiDAR, shortened to FWL, is one answer to those weaknesses. It captures the complete return waveform as a histogram, and that preserves rich reflection characteristics, including echoes from multiple paths and weak signals.

Transformer-based approaches have shown promise on this kind of data, but two bottlenecks stand in the way. The first is computational cost. Much of a waveform sits in low-intensity background regions, and turning all of it into tokens produces a great many redundant ones. The second is generalization. Annotated training data for FWL is scarce, so models struggle to carry what they learn to scenes they have not seen.

The authors address both bottlenecks with the same lever: the physical structure that is already inherent in FWL data. The intensity of a waveform says where the meaningful content is, and the geometry of how each ray is captured says what an honest variation of a scene must preserve. One method is built on each of those facts.

For cost, there is FWL-ToPM, a mechanism that prunes and merges tokens. It uses the distribution of intensity to decide which tokens to reduce, cutting the redundant background while keeping the critical waveform content.

For generalization, there is FWLAug, a data augmentation framework that acts as a regularizer. Its defining constraint is that it preserves waveform structure. For each ray it retains the Time-of-Flight waveform, and it keeps the frustum geometry intact. The goal is to improve generalization to unseen scenes without generating training examples that break the physics of the sensor.

Both are evaluated on the Ghost-FWL benchmark. FWL-ToPM delivers a speedup of more than eight times, reaching real-time throughput, and at the same time gains nearly nine points of F1 over the prior state of the art. The authors describe this as the first real-time FWL Transformer that also improves accuracy. FWLAug improves performance further, both on the standard benchmark and on an out-of-distribution test set made of unseen scenes, which the authors also make publicly available.

Throwing tokens away made this model faster and, at the same time, more accurate. The thread running through both methods is that nothing is chosen blindly: the data's own physics, its intensity and its ray geometry, decides what is kept and what is varied. Together the two methods are meant to push FWL-based perception toward practical deployment.