Red-Teaming the Agentic Red-Team

Research Overview

This paper presents the first comprehensive security analysis of agentic systems designed for offensive security operations (e.g., automated penetration testing). As these tools become increasingly commoditized and integrated into security workflows, they introduce significant new attack vectors. The authors demonstrate that an adversary controlling a target system can manipulate these agents to gain full control over the operator's machine, effectively turning the security tool into a weapon against its own user.

The Agentic Kill Chain

The authors introduce a specialized cyber kill chain that models how an attacker progresses from initial interaction to full system compromise. The process typically begins with the agent interacting with an adversary-controlled honeypot. By using contextual deception and reward hacking—rather than traditional prompt injection—the attacker tricks the agent into downloading and executing malicious payloads. Once code execution is achieved within the agent's worker environment, the attacker exploits weak system segmentation and poor privilege management to escalate access, establish persistence, and eventually escape the sandbox to compromise the host machine.

Security Implications

The study highlights that current agentic architectures are fundamentally insecure because they often rely on the optimistic assumption that LLMs can be hardened against manipulation. The authors argue that because these systems operate with high-impact capabilities like arbitrary code execution and network access, they must be designed under the assumption that the LLM will behave maliciously. They propose a robust architecture based on the principle of containment and least privilege, which aims to minimize the blast radius of an untrusted agent, ensuring that the system remains secure even if the underlying LLM is successfully manipulated.