Robust Context-Aware Detection of Malicious Instructions in Text

As LLM-based agents become more autonomous, they are increasingly vulnerable to indirect prompt injection (IPI) attacks, where malicious instructions are embedded in untrusted data sources. The authors investigate how to build a lightweight, robust detector that can segment malicious instructions from benign text while remaining resilient to adaptive evasion attacks that attempt to bypass security filters.

The authors propose Context-Aware Detection (CAD), a sentence-level classification framework. Unlike previous methods that analyze sentences in isolation, CAD constructs a contextual representation by concatenating a sentence's individual embedding with its marginal impact on the query-response pair. To ensure robustness against adaptive attackers, the authors implement two adversarial training (AT) strategies:

1. Feature-space AT: Uses projected gradient descent in the embedding space to approximate evasions.
2. Realizable AT: Uses LLM-based paraphrasing to generate realistic, non-optimized evasion attempts.

Both methods are parameterized by an adversarial data ratio, allowing developers to explicitly tune the trade-off between maintaining high utility (keeping benign content) and achieving high security (blocking malicious content).

Findings

CAD demonstrates superior performance compared to existing state-of-the-art defenses across multiple benchmarks, including AgentDojo and AgentDyn. The study reveals that while feature-space adversarial training provides a useful signal, the realizable LLM-paraphrasing approach generally yields better results against adaptive attacks. Crucially, the authors find that the optimal balance between utility and security is highly sensitive to the specific application domain, suggesting that a one-size-fits-all defense is insufficient for complex agentic workflows.