Yong Yang, Xing Zheng, Huiyu Wu, Huangsheng Cheng, Xiaorong Shi, Jing Guo, Bo Yang, Yi Zhou, Xiangfan Wu, Zonghao Ying
6 min
Abstract
The fast growth of open-source AI infrastructure, from model serving engines and agent platforms to the Model Context Protocol (MCP) ecosystem and the language models themselves, has outpaced the security tooling available to defend it. We present AI-Infra-Guard, an open-source framework that organizes AI red teaming around a single observation: the attack surface of an AI agent is stratified across layers (infrastructure, protocol/tool, agent behavior, and model), and no single detection paradigm fits all of them. The framework therefore matches a paradigm to each layer, from deterministic rule matching over 75+ AI components and 1{,}400+ vulnerability rules, through LLM-driven agentic auditing of MCP servers and agent-skill packages and multi-turn black-box agent red teaming, to a jailbreak harness with 26+ attack operators over sixteen datasets. To our knowledge it is the only open-source framework to span all of these, including supply-chain auditing of the agent skills that increasingly extend AI agents. We release AI-Infra-Guard as open source so that \emph{layer-paradigm matching} can serve as a practical foundation for agent security and a shared base for the community to build on.
Alex: So AI-Infra-Guard is a unified toolbox that knows which tool to reach for depending on which part of the AI it's inspecting.
Sam: That's the goal. The fragmentation in current security practice—where different teams use different tools with no shared picture—leaves real gaps. By organizing these methods into one architecture, the framework is designed to give developers a coherent view of risk across the entire system.
Alex: But if the framework is making active decisions about which tests to run, doesn't that introduce its own risks? What stops it from running the wrong test, or missing something because it assumed a tool wasn't needed?
Sam: That's a fair concern, and the paper addresses it directly. The system uses what they call a "layer-adaptive" approach. Before running any test, it checks whether the relevant tool actually exists for that layer—they call this "capability awareness." And once a vulnerability is confirmed, it applies stop rules to avoid running unnecessary additional tests. The idea is to treat the security scan like a planned mission rather than an exhaustive random search.
Alex: So it's efficient by design—not just thorough, but targeted.
Sam: Right. And that targeting also helps with a deeper problem: making sure the system isn't just guessing. The researchers use something they call "objective anchoring." Wherever possible, they replace a subjective judgment from an AI with a hard, verifiable check—like looking for a specific digital marker in the code. If the marker is there, the vulnerability is confirmed. No interpretation required.
Alex: It's like using a fingerprint instead of a description. A description can be wrong; a fingerprint match is definitive.
Sam: Exactly. They also encode security knowledge directly into natural language instructions for the AI components—a technique the paper calls "Prompt-as-Rule." This helps the system distinguish between a genuine threat and a harmless interaction that merely looks suspicious on the surface.
Alex: Here's what I keep coming back to, though. If part of the scanner is itself an AI, analyzing potentially malicious code—isn't the scanner at risk of being manipulated by the very thing it's scanning?
Sam: That is one of the more serious concerns the authors raise. Their solution is to treat the scanner itself as a potential target. They use what's called a "path sandbox"—an isolated environment where suspicious code can run without being able to reach or affect the scanner's own reasoning. And critically, the system analyzes the output of that code rather than the input, which cuts off a common route for manipulation.
Alex: So they've essentially built the security tool to be suspicious of its own safety. That seems like a necessary design choice for anything that has to interact with untrusted data.
Sam: It is. And the paper is candid about the limits here. The higher-level layers of the framework rely on language models, which means some degree of non-determinism is unavoidable—the same input might not always produce the same output. That's a meaningful constraint, and it's one the authors acknowledge rather than paper over.
Alex: So what we have is a framework that's more structured and more self-aware than what came before, but not a complete solution. It's a meaningful step toward making AI infrastructure auditable in a systematic way.
Sam: That's a fair summary. The paper's contribution is less about solving every problem and more about establishing a coherent way to think about the problem—matching the right kind of scrutiny to the right layer, building in safeguards against the scanner being deceived, and giving developers a unified picture instead of a patchwork of disconnected tools. Whether that architecture holds up as AI systems grow more complex is a question the field will have to keep revisiting.
Alex: It's a reminder that as we build more capable AI systems, the work of securing them has to grow in sophistication too. Thanks for listening to ResearchPod.