FUSE: Frequency-domain Unification and Spectral Energy Alignment for Multi-modal Object Re-Identification

Multi-modal person re-identification (ReID) aims to match individuals across different sensor types, such as RGB, near-infrared (NIR), and thermal infrared (TIR). While deep learning models have advanced this field, they typically operate in the spatial domain. The authors argue that these spatial-domain models suffer from an inherent low-frequency bias, which prioritizes coarse attributes like color and illumination while neglecting the mid and high-frequency details—such as textures and geometric structures—that are essential for robust identity discrimination. This leads to incomplete spectral representations and unstable cross-modal alignment.

The authors propose FUSE, a framework that shifts the focus from spatial-domain fusion to frequency-domain modeling. FUSE treats multi-modal ReID as a two-stage process:

1. Spectral Decomposition Module (SDM): This module uses the Discrete Fourier Transform (DFT) to convert spatial features into the frequency domain. It then adaptively partitions these features into low, mid, and high-frequency subspaces using learnable radial masks. Each subspace is processed with specialized convolutions (dilated, standard, and depthwise) to capture its unique structural information before being reconstructed via the Inverse Discrete Fourier Transform (IDFT).
2. Cross-Modal Alignment Module (CAM): This module addresses the remaining spectral discrepancies between modalities. It uses multi-head cross-attention to allow each modality to aggregate complementary information from others, while a frequency-consistency regularization term enforces global spectral alignment across the different sensor inputs.

FUSE significantly outperforms existing state-of-the-art methods on standard multi-modal ReID benchmarks. Specifically, on the RGBNT201 dataset, the framework achieved a 9.1% improvement in mean Average Precision (mAP) and a 9.5% improvement in Rank-1 accuracy. These results suggest that explicitly modeling frequency-domain interactions provides a more interpretable and robust paradigm for integrating heterogeneous sensor data compared to traditional spatial-only approaches.