ResearchPod Summary
Medical vision-language models (VLMs) often struggle with clinical trustworthiness because they generate diagnoses without identifying the specific image regions that support their conclusions. This "black box" approach makes it difficult for clinicians to audit findings and increases the risk of hallucinations, particularly in brain MRI where precise lesion localization is critical. The authors ask whether an explicit, two-turn "mark-and-rethink" paradigm can improve diagnostic accuracy and reduce false positives in both in-distribution and out-of-distribution (OOD) clinical settings.
The authors introduce BrReMark, a framework that forces the model to follow a structured two-turn reasoning process. In the first turn, the model generates a hypothesis and explicitly marks a bounding box on the MRI. In the second turn, it re-examines the marked image to verify its initial finding. The training process is two-fold: first, supervised fine-tuning (SFT) teaches the model the required two-turn format; second, reinforcement learning (RL) uses a composite reward function—incorporating localization accuracy, semantic correctness, and clinical safety—to align the model with human diagnostic standards. To improve robustness against rare pathologies, the authors also implement a synthetic pathology injection pipeline that uses domain randomization to augment training data with realistic lesion patterns.
BrReMark significantly outperforms base models and other medical VLMs in both localization and diagnostic reasoning. On internal benchmarks, it achieved a 37.54% mAP50 for anomaly localization, compared to 0.74% for the base model. Crucially, the framework demonstrates strong OOD generalization; on the NOVA benchmark, it achieved a 45.7% reduction in false positives compared to state-of-the-art models. The results suggest that forcing the model to ground its reasoning in visual evidence is a highly effective strategy for mitigating hallucinations and improving the reliability of AI-assisted radiology.
This work addresses a fundamental barrier to the clinical deployment of medical AI: the lack of auditable reasoning. By providing a framework that mirrors the human diagnostic workflow—observing, localizing, and verifying—BrReMark provides a path toward AI systems that are not only accurate but also transparent and verifiable. The combination of synthetic data augmentation and reinforcement learning offers a scalable way to handle the long-tail distribution of rare diseases, which is a common failure point for existing medical models.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.