GAUSSIANFUSION: UNIFIED 3D GAUSSIAN REPRESENTATION FOR MULTI-MODAL FUSION PERCEPTION

Autonomous driving perception systems often struggle with the information loss inherent in discretizing multi-modal sensor data (like camera images and LiDAR point clouds) into fixed-resolution Bird's-Eye View (BEV) grids. The authors investigate whether a continuous 3D representation, specifically 3D Gaussian Splatting, can better preserve fine-grained spatial and semantic details while enabling more natural cross-modal alignment and fusion.

GaussianFusion introduces a novel framework that represents 3D scenes as a collection of 3D Gaussians rather than discrete voxels. The process begins with a forward-projection-based initialization module that maps camera and LiDAR features into a shared 3D Gaussian space. A shared cross-modal Gaussian encoder then iteratively refines these Gaussian properties—mean, scale, and rotation—using a deformable attention mechanism that leverages the geometric priors of the Gaussians themselves. Finally, the model uses a Gaussian-to-voxel pooling module to transform these continuous representations into a format suitable for standard task-specific perception heads, such as 3D object detection and semantic occupancy prediction.

GaussianFusion demonstrates superior performance across multiple benchmarks. On the nuScenes dataset, it outperforms the established BEVFusion baseline by 2.6 NDS in 3D object detection. For 3D semantic occupancy, the model achieves a 1.55 mIoU improvement over the previous state-of-the-art while utilizing only 30% of the Gaussian count and delivering a 450% speedup. The authors show that the Gaussian representation effectively captures object boundaries and fine textures that are typically lost in grid-based quantization.

This work provides a compelling alternative to the dominant BEV paradigm in autonomous driving. By treating the scene as a continuous, adaptive Gaussian mixture model, the framework allows for more flexible and efficient multi-modal fusion. The task-agnostic nature of the unified Gaussian representation suggests it could serve as a robust foundation for a wide range of future 3D perception tasks.