A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy

Vision-language models (VLMs) are increasingly used for medical decision support, yet their tendency to hallucinate—generating plausible but factually incorrect responses—poses a significant safety risk. While many detection methods have been validated in radiology, their performance in gastrointestinal (GI) endoscopy remains largely unverified. This study benchmarks nine hallucination detection methods across five diverse VLMs (including MedGemma and LLaVA variants) using the Gut-VLM dataset, which contains 4,392 expert-verified VQA pairs.

The authors categorize detection methods by their level of model access: black-box (using only generated text), gray-box (using token-level probabilities), and white-box (using internal hidden states). They evaluate these methods on a unified pipeline where hallucination labels are derived using the GREEN model, which compares VLM outputs against expert-verified ground truth. The white-box method, ReXTrust, is trained on the Gut-VLM training set to classify responses based on internal hidden-state representations.

The study reveals that white-box access provides a substantial and consistent performance advantage, with ReXTrust outperforming the strongest non-white-box alternative by an average of 19.5 AUC points. Notably, while black-box and clustering-based gray-box methods often collapse to near-chance performance on general-purpose models like LLaVA-v1.6-7B, ReXTrust maintains strong discriminative power. Among non-white-box approaches, token-level gray-box statistics (such as MaxEnt and MaxProb) are the most reliable alternatives. The authors also identify a systemic failure mode termed 'confident confabulation,' where models consistently generate the same incorrect answer, causing both consistency-based and uncertainty-based detection methods to fail.