MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG

As RAG systems become increasingly agentic and multimodal, their attack surfaces have expanded beyond simple prompt injection to include retrieval poisoning, image-based instructions, and tool-manipulation attacks. Current red-teaming methods are often surface-specific, prone to recycling known attack templates, and lack the ability to generalize across the heterogeneous components of an agentic pipeline. This paper asks: can a unified, memory-guided planning framework discover novel, diverse vulnerabilities across these varied attack surfaces while maintaining high efficiency?

The authors introduce MIRROR (Memory-Informed Red-teaming with Retrieval-Restricted Optimization and Rollouts). MIRROR treats attack generation as an adversarial planning problem using Monte Carlo Tree Search (MCTS). It incorporates an episodic memory bank of successful attack traces, which the system uses to inform its search priors. A key innovation is the deterministic Novelty Gate, which rejects any candidate attack that matches previously seen prompts or retrieved context under specific normalization rules. This ensures that the system discovers genuinely new attack patterns rather than simply duplicating existing ones. The framework is evaluated on a new, large-scale dataset, ART-SAFEBENCH, which covers four distinct attack surfaces: text poisoning, image injection, direct-query attacks, and orchestrator tool manipulation.

MIRROR demonstrates superior performance and consistency compared to specialized baselines. While existing methods often collapse when applied to surfaces they were not designed for—such as suffix optimization achieving high success on text but failing on direct queries—MIRROR maintains high attack success rates (ASR) across all tested surfaces. Specifically, it achieved 76% ASR on image poisoning and 97% ASR on orchestrator attacks, the latter at half the query cost of baselines. Furthermore, MIRROR exhibits the lowest cross-surface variance, indicating it is a robust, general-purpose tool for identifying vulnerabilities in complex agentic RAG architectures.