Human and AI collaboration for pulmonary nodule segmentation

High-quality segmentation of pulmonary nodules is essential for AI-driven clinical workflows, yet manual annotation by radiologists is labor-intensive and difficult to scale. While foundation models like the Segment Anything Model (SAM) show promise, they often struggle with medical images when provided with realistic, non-idealized prompts. This study investigates whether a human-in-the-loop framework can bridge this gap, allowing diverse annotators to collaborate with AI to produce high-quality masks efficiently.

The authors developed Hi-Seg, a framework that integrates human interaction with SAM. Instead of relying on pre-existing ground-truth masks for prompting, human annotators provide iterative feedback through a simple interface. By clicking within a nodule to include it or on background pixels to exclude them, users guide the model toward a refined mask. The study validated this approach using 1,179 chest CT scans across 12 centers, comparing the performance of five participants—ranging from a senior radiologist to non-medical personnel—against state-of-the-art deep learning models and various SAM variants.

Hi-Seg achieved a mean Dice score of approximately 85%, significantly outperforming both specialized deep learning architectures and SAM variants that relied on pseudo-human prompts. A key finding was that the iterative, closed-loop interaction allowed annotators to learn effective prompting strategies over time, leading to faster convergence on accurate masks. Notably, briefly trained non-medical annotators achieved performance levels comparable to junior medical students, and the framework reduced annotation time for these groups by roughly 30%. While Hi-Seg improved accuracy across most nodule types, it was particularly effective at narrowing the performance gap between experts and non-experts, though expert oversight remains necessary for complex cases like ground-glass opacities.