SARFA: Segment Anything with Radiomic Feature Alignment

Medical image segmentation is often hindered by inherent ambiguity—such as unclear lesion boundaries or inter-rater disagreement—where multiple valid segmentations exist for a single image. Standard models often struggle with this because they are deterministic, producing only one output. This study asks whether incorporating radiomic features (quantitative descriptors of texture and morphology) as a supervisory signal can better guide a foundation model like SAM to produce more clinically accurate and diverse segmentation candidates.

The authors introduce SARFA (Segment Anything with Radiomic Feature Alignment), a framework that adapts the Segment Anything Model (SAM) for ambiguous medical imaging. SARFA generates multiple candidate masks using SAM's built-in multimask decoding. To evaluate these candidates, the authors use PyRadiomics to extract radiomic features from both the predicted masks and the ground truth. They then calculate the Fréchet Radiomic Distance (FRD) to quantify the similarity between these feature distributions. Finally, they employ Direct Preference Optimization (DPO) to rank the candidate masks, training the model to favor outputs that minimize the FRD relative to the ground truth.

SARFA outperforms ten state-of-the-art baselines, including P2SAM and various probabilistic U-Net variants, on both CT (LIDC-IDRI) and MRI (BraTS2017) datasets. The results show significant improvements in Generalized Energy Distance (GED) and FRD, suggesting that the model produces a distribution of plausible masks that better aligns with the anatomical and textural characteristics of expert annotations. Qualitative analysis indicates that SARFA is more robust at capturing lesion boundaries where previous models often failed.

By moving beyond simple pixel-level overlap (like standard Dice loss), SARFA provides a way to incorporate domain-specific clinical knowledge into foundation models. This approach effectively addresses the "one-to-many" nature of medical image interpretation, offering a more flexible and accurate tool for radiologists dealing with ambiguous diagnostic targets.