DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA

This paper investigates the disconnect between traditional natural language evaluation metrics (like ROUGE and BERT scores) and the actual factual grounding of Retrieval-Augmented Generation (RAG) systems. Specifically, the authors explore whether state-of-the-art frontier models truly utilize retrieved scientific documents to generate answers or if they rely on their own internal knowledge, potentially leading to hallucinations or ungrounded citations.

The authors evaluate several RAG architectures on the CLEF 2026 LongEval Task 4, which involves answering scientific queries using a provided set of ten documents. They compare a baseline hybrid retriever against a corrective pipeline that integrates:

• Corrective RAG (CRAG): A pre-generation filtering step that uses a cross-encoder to discard irrelevant chunks.
• CiteFix: A post-generation pruning step that removes citations not supported by the generated text.

These pipelines are benchmarked against frontier models (GPT-5.5 and Claude Opus 4.7) using both official task metrics and a reference-free RAGAs framework, which uses an LLM-as-a-judge to measure 'Global Faithfulness' and 'Citation Faithfulness.'

Findings

The study reveals a significant divergence in performance metrics. Frontier models dominate the official leaderboard, achieving high lexical and semantic similarity to gold answers. However, RAGAs diagnostics show these models have low citation faithfulness, indicating they often generate relevant-sounding answers without actually grounding them in the provided context. In contrast, the corrective pipeline (CRAG+CiteFix) forces the system to abstain when evidence is insufficient, resulting in lower ROUGE/BERT scores but significantly higher structural honesty and factual grounding.