Cost-Pragmatic Quality Gating and Selection–Fusion Multi-Model Combiners for BioASQ Phase A+/B

This paper addresses two primary challenges in the BioASQ biomedical question-answering task: optimizing the computational cost of retrieval-augmented generation (RAG) and determining the most effective strategy for combining answers from multiple language models (LLMs). The authors investigate whether aggressive re-retrieval is always necessary and how to best aggregate multi-model outputs to maximize performance across different evaluation metrics.

The authors introduce a two-stage retrieval architecture that unions a hybrid first-stage (dense BGE, BM25, and reciprocal-rank fusion) with an agent-driven pipeline. To manage costs, they implement a BGE cross-encoder quality gate that categorizes questions by retrieval confidence. Only questions flagged as having severe or borderline coverage are subjected to expensive re-retrieval. For answer generation, the authors propose a selection–fusion decomposition. They argue that while LLM-as-judge methods are effective for selection-dominated metrics (like yes/no), they are structurally limited by the per-question oracle. Conversely, they implement a deterministic synonym-union resolver that aggregates list answers from multiple models to maximize recall, accepting a potential precision penalty.

The cost-pragmatic re-retrieval policy significantly outperformed the skill-strict baseline on list F1 and precision while saving approximately 12% in retrieval costs. The study confirms that while LLM-as-judge approaches are sufficient for metrics like yes/no, they are insufficient for list-based recall. The synonym-union resolver successfully increased list recall on every head in the Task 13B 2025 dataset. On the preliminary Task 14B 2026 leaderboard, the system achieved first place on the combined-exact aggregate across multiple batches and secured the top position in the Phase B b3 ideal track.