Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance

Multi-modal object Re-Identification (ReID) aims to match objects across different camera views by combining information from multiple sensors (e.g., RGB, NIR, and thermal). Existing methods often struggle with two major issues: they fail to effectively suppress background noise and they lack a mechanism for comprehensive tri-modal feature alignment, often relying on computationally expensive pairwise fusion.

The authors propose PRISM, a framework built on three core innovations:

• Prompt-S6 (PS6): A lightweight module based on State Space Models (SSMs). Unlike traditional attention mechanisms that have quadratic complexity, PS6 uses decoupled prompt conditioning to allow external signals (like semantic masks) to control how information is written into and read from the latent state, maintaining linear complexity.
• Semantic-Driven Token Pruning (SDTP): This component uses pre-computed semantic masks to guide the model. It employs a Target-Aware Feature Enhancement (TAFE) stage to align semantic and spectral features, followed by a Tail Drop Module (TDM) that suppresses tokens likely associated with background clutter.
• Progressive Fusion Network (PFN): A three-stage pipeline that integrates intra-modal modeling, inter-modal interaction, and final tri-modal aggregation to ensure that all three modalities contribute effectively to the final representation.

PRISM demonstrates that by using semantic priors to guide token pruning, the model can focus its capacity on discriminative foreground regions, significantly improving robustness in complex scenarios. The use of PS6 allows the model to perform these interactions with fewer parameters and lower computational overhead compared to standard attention-based architectures. Experimental results across four multi-modal benchmarks confirm that PRISM achieves superior accuracy and efficiency compared to existing state-of-the-art methods.