Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build **ToolHazard-Bench** for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study called ToolHazard, which explores how AI agents interact with the world—and the security risks that come with that.
Sam: Right. AI assistants are increasingly being given the ability to use tools on our behalf—things like reading our email, managing our calendar, or editing files. And here's the problem: the moment an AI starts reading content from the outside world, that content can be weaponised against it. The study focuses on a specific kind of attack called "indirect prompt injection," where malicious instructions are hidden inside the very data the agent is supposed to process.
Alex: So it's like if you asked your assistant to read your emails and summarise them—but one of those emails secretly contained instructions telling your assistant to forward everything to a stranger?
Sam: That's exactly the scenario. And what makes it tricky is that the AI has no obvious way to tell the difference between a legitimate instruction from you and a hidden instruction buried in an email someone else wrote. It just sees text, and it tries to follow it.
Alex: So how do you even train an AI to defend against that?
Sam: That's the central contribution of this paper. Traditionally, researchers test AI agents on fixed, hand-crafted scenarios—a small set of situations that humans wrote out manually. The problem is that attackers are creative, and those static test sets go stale quickly. What this study proposes is a "security simulation factory"—a system that automatically generates thousands of realistic, varied environments for testing agents, rather than relying on humans to build each one by hand.
Alex: So instead of a handful of test cases, you get a whole library of them, built automatically.
Sam: Exactly. And the environments aren't trivial. They represent real software systems—things like a calendar app, a file manager, a database. The factory builds a blueprint for each one, writes the actual working code, and then uses what the paper calls a "dual-agent" system to verify it. One agent tries to break the environment, looking for flaws, while another checks whether the rules are being followed correctly.
Alex: So it's quality-controlling its own output. That's a sensible design. But once you have these environments, how do you find where the attacks can actually happen?
Sam: That's where the "Attacker Agent" comes in. It analyses the code of each environment and looks for what the paper calls "attack points"—specific places where an outside party could write text that the AI will eventually read. Think of an email body, a comment field, a document someone uploads. The attacker agent maps the path from where the malicious text gets written to where the AI agent will encounter it.
Alex: So it's essentially scouting the building for unlocked windows before a burglar does.
Sam: That's a useful way to put it. Once those entry points are identified, the system automatically plants hidden instructions—"hijack tasks"—inside them. Then it watches to see whether the target agent gets tricked into following those instructions instead of the legitimate ones it was given.
Alex: And I'd imagine the results aren't always clean. Does it matter where in the content the attack is placed?
Sam: It does, and that's one of the more interesting findings. The experiments show that attacks tend to be more effective when the malicious instruction appears early in the agent's overall workflow, and when it's placed toward the end of the specific piece of content the agent is reading. The paper suggests that both timing and placement are meaningful factors in how easily an agent can be confused.
Alex: So the AI is more vulnerable at certain moments—like being distracted at the start of a task?
Sam: That seems to be the implication, yes. It points to the idea that context matters enormously. An agent processing its first piece of information is in a different state than one that's already deep into a task and has more context to work with.
Alex: So once you've identified these vulnerabilities, how do you actually fix them? How do you train the agent to resist?
Sam: The paper uses a technique called reinforcement learning. The basic idea is similar to how you'd train any system through feedback. The agent attempts a task, and the system gives it a signal—a reward if it completed the legitimate task correctly, and a penalty if it got tricked into following a hijack instruction. Over many thousands of attempts across many different simulated environments, the agent gradually learns to tell the difference.
Alex: So it's practising in a safe space, making mistakes, and adjusting. Like a flight simulator for AI security.
Sam: That's a good analogy. The specific training method the paper uses is called GRPO—it works by having the agent generate several different responses to the same situation and then comparing them. The agent learns not just to avoid being tricked, but to reason about which instructions are legitimate and which aren't. The goal is to make it more discerning, not just more hesitant.
Alex: That distinction matters. You don't want an agent that just refuses to do anything because it's paranoid. You want one that can still be useful.
Sam: Precisely, and that's what the researchers measured. They tracked what they call the "benign task completion rate"—essentially, how well the agent still performs its normal, legitimate work after security training. The finding is that the trained agents remained just as capable at their everyday tasks. The security improvements didn't come at the cost of usefulness.
Alex: That's a meaningful result. It suggests these two goals—being safe and being helpful—aren't necessarily in conflict.
Sam: That does seem to be what the paper argues. And the broader point is about scale. By automating the generation of these complex test environments, the researchers have built a system that can keep pace with evolving attack strategies, rather than waiting for humans to manually document each new threat. It's a move toward a more proactive approach to AI safety—one where the testing infrastructure grows alongside the capabilities of the agents themselves.
Alex: So the factory doesn't just test what we know is dangerous today. It builds the tools to discover what might be dangerous tomorrow.
Sam: That's the ambition. Whether it fully delivers on that promise will depend on how well these simulated environments reflect the complexity of real-world systems. But as a framework for thinking about AI agent security, it's a meaningful step forward.
Alex: Thanks for walking us through it. And thanks to everyone listening to ResearchPod.