Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework

Retrieval-Augmented Generation (RAG) systems are increasingly vulnerable to knowledge corruption, where adversaries inject poisoned documents into the retrieval corpus to manipulate LLM outputs. Existing defenses often rely on simplistic keyword filtering or unrealistic assumptions about the proportion of malicious content. This paper introduces SecureCollaRAG, a framework designed to identify and filter out poisoned documents by treating the multi-source retrieval process as a Byzantine-robust aggregation problem.

SecureCollaRAG employs a three-stage verification process to ensure document integrity:

1. Distributed Knowledge Graph Construction: The system builds a dynamic graph where documents are nodes and edges represent inter-document similarity, allowing the system to model relationships across multiple independent sources.
2. GNN-based Credibility Scoring: A Graph Neural Network (GNN) processes the graph topology and node features to compute credibility scores, identifying patterns indicative of malicious behavior.
3. Verification-based Aggregation: Documents with low credibility scores are discarded. For documents in an 'uncertain' range, the system uses a secondary commonsense verification step via prompt engineering to assess logical consistency before final aggregation.

The authors demonstrate that SecureCollaRAG effectively maintains robustness against knowledge corruption attacks, even under non-IID data distributions where data varies significantly between sources. By integrating dynamic graph learning with multi-source validation, the framework provides a scalable defense that does not rely on simplistic statistical assumptions. This is particularly important for high-stakes environments, such as clinical or institutional settings, where even minor data poisoning can lead to severe reliability issues in AI-generated responses.