Sparse-Aware Vector Quantization for Bandwidth-Efficient Collaborative 3D Semantic Occupancy Prediction

Collaborative perception allows autonomous vehicles to share information to overcome sensor limitations and occlusions. However, 3D semantic occupancy prediction requires high-dimensional spatial data, creating a severe trade-off between perception accuracy and communication bandwidth. This paper asks: how can we perform collaborative 3D semantic occupancy prediction without the prohibitive bandwidth costs of transmitting dense 3D features?

The authors propose the Vector Quantization Semantic Occupancy Prediction (VQSOP) framework. It introduces two key innovations:

1. Sparse-Aware Vector Quantization (SAVQ): Instead of sending dense features, the system uses a confidence-based selector to identify informative 3D regions. These features are then quantized into discrete indices using a shared, learnable codebook. Only these compact indices are transmitted, drastically reducing bandwidth usage.
2. Dual-Branch Adaptive Spatial Refinement (ASR): To ensure high-quality reconstruction after fusion, this module uses a parallel architecture. A local branch captures fine-grained geometric details, while a context branch uses dilated convolutions to model long-range semantic dependencies. These are combined using spatially adaptive weighting to produce a coherent 3D scene representation.

VQSOP achieves state-of-the-art performance on the Semantic-OPV2V dataset. It outperforms existing methods by 4.10% in mIoU under collaborative settings while reducing communication volume by up to 82x compared to dense transmission methods. The ablation studies confirm that the SAVQ mechanism effectively filters out task-irrelevant background noise, and the ASR module is essential for recovering fine-grained structures like poles and guard rails that are often lost in standard collaborative fusion.