Learning Structurally Consistent Representations for Multi-View Radar Semantic Segmentation

Radar sensors are essential for autonomous driving due to their robustness in adverse weather, yet their inherent sparsity and noise make dense semantic segmentation difficult. Conventional methods often rely on grid-based convolutions or pairwise interactions, which fail to capture the complex, higher-order structures formed by multiple radar returns originating from the same physical object. This paper asks how to better model these higher-order dependencies and align heterogeneous radar views that exhibit varying measurement densities.

The authors propose HyperRadar, an end-to-end framework that integrates three key components:

• Hypergraph Refinement: Instead of standard convolutions, the model uses learnable hypergraphs where a single hyperedge connects multiple spatially and spectrally related radar responses. This allows the network to capture object-level structure and maintain coherence despite sparse or fragmented data.
• Unbalanced Optimal Transport (UOT): To handle the challenge of aligning radar views (e.g., Range-Doppler and Range-Angle) that have different measurement densities and partial overlaps, the authors use UOT. Unlike standard optimal transport, UOT allows for mass variation, making it robust to the view-dependent sparsity characteristic of radar.
• Adaptive Attention: An attention mechanism dynamically weights radar responses, emphasizing informative features while suppressing noise before the final segmentation decoding.

The model is trained using a combination of supervised segmentation and cross-view consistency regularisation, which enforces agreement between different radar projections along their shared dimensions.

HyperRadar demonstrates significant improvements over existing radar-specific baselines on the CARRADA and RADIal benchmarks. On CARRADA, it achieves 63.8% mIoU in the Range-Doppler (RD) view and 44.4% mIoU in the Range-Angle (RA) view, outperforming previous state-of-the-art methods. On the RADIal benchmark, the model achieves 83.4% mIoU, setting a new performance standard. The ablation studies confirm that both the hypergraph refinement and the UOT alignment contribute to these gains, with the full model providing the most robust performance across sparse foreground classes like pedestrians and cyclists.