Towards Real-Time and Adaptable LiDAR Scene Completion

Can LiDAR scene completion achieve the real-time inference speeds required for autonomous driving without sacrificing geometric accuracy, and how can we avoid the computational bottlenecks of slow generative models and heavy point-neighborhood operators?

Existing methods either use slow iterative diffusion starting from random noise or rely on single-pass networks with fixed-noise perturbations and expensive point-neighborhood operators like farthest point sampling (FPS) and k-nearest neighbors (k-NN). RapidLiDAR addresses these limitations through an initialize-and-refine paradigm built around two core components:

1. Adaptive Initialization Module: Instead of using random noise or fixed-variance perturbations, it predicts spatially varying, data-driven displacements for input points. Points in sparse or occluded regions are displaced further to fill gaps, creating a coarse initialization tailored to local geometry.

2. Multi-Scale Reconstruction Module: It extracts multi-scale 3D voxel features and projects them into 2D Bird's-Eye-View (BEV) maps. By replacing FPS and k-NN with multi-scale deformable cross-attention over these BEV maps, the network efficiently queries global context to refine point positions into a complete scene.

The architecture is trained end-to-end using Chamfer Distance loss on outdoor driving benchmarks.

Evaluated on the SemanticKITTI and KITTI-360 datasets, RapidLiDAR achieves scene completion accuracy on par with the current state of the art. Crucially, it processes a full outdoor LiDAR scene in 0.1 seconds, matching the 10 Hz acquisition rate of standard automotive sensors and outperforming the fastest prior baseline by a factor of 2.3.

Real-time 3D scene completion is a critical bottleneck for downstream perception tasks in autonomous vehicles, such as collision avoidance and motion planning. By eliminating iterative sampling and scaling-heavy point operations, RapidLiDAR demonstrates that high-fidelity geometric completion can run fast enough for live, safety-critical driving applications.