ARTEMIS: Agent-guided Reliability-aware Temporal Mask Evolution for Imperfectly Supervised Video Polyp Segmentation

How can we effectively train video polyp segmentation (VPS) models when pixel-level annotations are sparse (points/scribbles) or only available for a small subset of frames? The authors seek to overcome the limitations of existing methods, which often suffer from noisy pseudo-labels, boundary leakage, and a failure to leverage temporal consistency in video data.

The authors propose ARTEMIS, a two-stage framework that follows a complete-then-learn paradigm. In the first stage, the model uses a vision-language agent (Qwen2.5-VL) to debate and judge the reliability of coarse masks generated by SAM2. These masks are then used as temporal anchors and propagated bidirectionally across the video to refine unreliable or unlabeled frames. In the second stage, the segmenter is trained using a robust learning strategy that incorporates a Reference Prototype Transport Module (RPTM) and a reliability-aware loss function to suppress noise and maintain temporal identity consistency.

ARTEMIS demonstrates that dense, noisy mask prompts are significantly more effective than sparse point prompts for guiding segmentation in weakly discriminative medical images. By using a debate-and-judge agent to select reliable temporal anchors, the framework successfully injects foundation-model priors into the training process. Experiments on the SUN-SEG and CVC-ClinicDB-612 datasets show that ARTEMIS achieves state-of-the-art performance across various imperfect supervision settings, effectively handling challenges like motion blur, weak contrast, and specular highlights.

Manual annotation of medical videos is prohibitively expensive and time-consuming. By establishing a unified, robust framework for imperfectly supervised VPS, ARTEMIS reduces the reliance on dense pixel-level labels while simultaneously improving segmentation accuracy. This approach provides a scalable path for deploying computer-aided diagnostic tools in clinical settings where fully annotated datasets are unavailable.