ResearchPod Summary
Multimodal large language models (MLLMs) frequently suffer from object hallucination, where they confidently describe objects that are entirely absent from the input image. This flaw undermines their reliability in high-stakes deployment settings such as assistive image description and medical report generation. Existing remedies largely fall into the short-caption regime, operating as decoding-time interventions that fail to scale when captions become longer and more detailed. Standard supervised fine-tuning on detail-rich corpora elongates descriptions, but leaves a high absolute rate of hallucinated objects. This paper introduces Dual-Stream Cross-Anchor Correction (DSCC), a novel training-time framework designed to eliminate object hallucinations without sacrificing caption length or richness of detail.
Unlike post-processing decoding methods or external vision-encoder constraints, DSCC places grounding constraints directly inside the language model during fine-tuning through two auxiliary information streams:
A two-stage curriculum gate couples these streams progressively, ensuring stable mixed-precision training and preventing the cognitive cross-attention from disrupting the perception anchors before they converge.
Experiments conducted under a unified backbone and scoring protocol demonstrate that DSCC is the only method reaching the desirable long-caption, low-hallucination region. Specifically, the model produces captions roughly 1.9 times the baseline length while maintaining 88.19% precision per object mention. Controlled ablations reveal a striking synergy between the two streams: while the perception stream alone degrades discriminative precision on its own, it reverses sign and yields massive gains when stacked onto the cognition stream.
While DSCC proves highly effective across general object-centric evaluation benchmarks, testing on out-of-domain datasets reveals a predictable domain-conditionality. The demonstrated synergy relies strictly on the semantic domain bound by the CLIP-COCO anchor mapping, failing when confronted with abstract inputs like charts and optical illusions.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.