RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems

As agentic AI systems—which integrate LLMs with tools, memory, and multi-step reasoning—become more prevalent, they introduce complex security risks beyond traditional prompt injection. Current security evaluation methods are often fragmented, tied to specific implementations, or limited to black-box testing. This paper asks: How can we create a unified, scalable, and implementation-agnostic methodology to evaluate the security of diverse agentic architectures?

The authors introduce RIFT-Bench, a two-phase methodology for dynamic red-teaming.

1. Discovery Phase: The system automatically analyzes the target codebase to generate a "NodeSpec," a hierarchical graph representation that maps the agent's components, tools, memory, and internal communication channels. This step allows the framework to understand the system's structure regardless of the underlying framework or architecture.
2. Scanning Phase: Using the NodeSpec, the framework selects and instantiates relevant adversarial probes from a library of over 100 templates. These probes are injected into the system to test specific vulnerabilities, such as tool misuse, memory poisoning, or privilege escalation. The system then traces the execution to evaluate the impact of these attacks.

The researchers validated RIFT-Bench across 45 heterogeneous agentic systems spanning various frameworks and domains (e.g., finance, medical, and personal assistants). By generating over 10,000 distinct attack tests, they demonstrated that the framework effectively adapts to different architectures and provides a consistent way to measure vulnerability. The results show that RIFT-Bench can not only identify security flaws but also support the evaluation of mitigation strategies, making it a robust foundation for ongoing security research in agentic AI.