Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion

Traditional fully-supervised Salient Object Detection (SOD) models require expensive, pixel-level manual annotations. This paper addresses the challenge of achieving high-performance RGB-D SOD using only sparse, cost-effective scribble annotations, which typically lack the structural detail necessary for accurate saliency map generation.

The authors introduce a two-stage framework:

1. SAM-PAG (SAM-driven Pseudo Annotation Generation): This module leverages the Segment Anything Model (SAM) to convert sparse scribbles into dense pixel-level pseudo-labels. It employs a dual-branch structure—an image transformation branch (handling scaling, rotation, and flipping to overcome SAM's sensitivity to input variations) and a prompt extension branch (using superpixel propagation to expand potential object regions). A consistency-based fusion method integrates these outputs to produce reliable dense supervision.
2. S²Diff (State Space Interaction-based Conditional Diffusion Model): This module acts as the saliency detector. It uses a conditional diffusion process to iteratively refine noisy saliency maps. It incorporates a Cross-modal Conditional Generation Module (CCGM) to interweave RGB and depth features using frequency-domain integration and state space interaction, and a Context Injection Module (CIM) to mitigate noise and enhance object boundaries during the denoising process.

The proposed method effectively bridges the gap between weakly-supervised and fully-supervised performance. By utilizing SAM to generate dense pseudo-labels, the model overcomes the limitations of previous scribble-supervised approaches that relied on rougher, model-predicted labels. Experimental results across seven standard datasets demonstrate that the combination of SAM-PAG and S²Diff outperforms existing scribble-supervised methods and achieves competitive results compared to fully-supervised benchmarks.