Mixture of Cognitive Experts in Large Vision-Language Models

Large Vision-Language Models (LVLMs) often struggle with complex reasoning and hallucination because they treat all queries with a uniform perception-reasoning pipeline. This paper asks whether integrating specialized computer vision experts through a structured, cognitively-grounded protocol can improve model performance, interpretability, and verifiability.

The authors propose an evidence-driven framework that operates in three main stages:

1. Expert Routing: An LLM-guided router selects a subset of specialized vision experts (e.g., for OCR, object detection, or segmentation) relevant to the user query.
2. Literal Evidence Summary: Expert outputs are decomposed into atomic, citeable evidence statements, creating a unified and verifiable evidence store.
3. Bloom Verbalization: The model generates a staged reasoning trace following Bloom’s taxonomy (e.g., List, Explain, Illustrate, Compare, Hypothesize). This trace acts as an intermediate scaffold that is fed to the final LLM to guide its reasoning process.

Additionally, the authors include a Reasoning Trace Module that parses these outputs to provide quantitative insights into how the model uses evidence and at what cognitive depth it operates.

The proposed architecture, evaluated on several vision-language benchmarks, demonstrates consistent improvements in perception and reasoning tasks compared to standard LVLM baselines. By forcing the model to ground its reasoning in explicit, atomic evidence, the framework reduces hallucinations and provides a clear, inspectable path from visual input to final answer. Furthermore, the analysis reveals that most queries in current benchmarks primarily require basic recall or understanding, rather than higher-order analytical reasoning, highlighting a potential gap in current evaluation datasets.