ResearchPod Summary
Large Vision-Language Models (LVLMs) frequently suffer from object hallucination, where they generate descriptions of objects that are not present in the input image. While previous detection methods often rely on auxiliary models or complex visual grounding verification, this paper investigates whether LVLMs already possess internal signals for truthfulness that can be surfaced directly through the language model (LM) head without adding inference-time overhead.
The authors observe that while real and hallucinated object tokens are clearly separable in the model's internal hidden representations, this information is largely lost in the final output probability distributions. To bridge this gap, they introduce TruthLens, a self-evaluation framework. They repurpose a rarely-used special token (e.g., <unk> or <|image_pad|>) as a reference signal. During a parameter-efficient fine-tuning phase (using LoRA), the model is trained to adjust the log-probability of this special token at each object-mention position. A mean squared error (MSE) objective drives this probability toward a high value for real objects and a low value for hallucinated ones, while a KL-divergence constraint ensures the model's original generative capabilities are preserved.
TruthLens demonstrates state-of-the-art performance in detecting object hallucinations across multiple LVLMs, including LLaVA and Qwen2.5-VL architectures. On the MS-COCO benchmark, the method significantly outperforms existing detection baselines, achieving a 17% improvement in AUROC for the Qwen2.5-VL-7B model. Notably, the framework generalizes effectively to datasets with larger label spaces (such as Object365) despite being trained on a limited set of categories, and it introduces zero additional computational cost during inference.
This approach provides a lightweight, efficient, and model-agnostic way to improve the reliability of LVLMs. By enabling models to perform self-evaluation of their own outputs, TruthLens offers a path toward more trustworthy AI systems in safety-critical applications like medical imaging and autonomous driving, without the latency penalties associated with external verification models.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.