Large language models (LLMs) are increasingly applied in specialized domains such as finance and healthcare, where they introduce unique safety risks. Domain-specific datasets of harmful prompts remain scarce and still largely rely on manual construction; public datasets mainly focus on explicit harmful prompts, which modern LLM defenses can often detect and refuse. In contrast, implicit harmful prompts-expressed through indirect domain knowledge-are harder to detect and better reflect real-world threats. We identify two challenges: transforming domain knowledge into actionable constraints and increasing the implicitness of generated harmful prompts. To address them, we propose an end-to-end framework that first performs knowledge-graph-guided harmful prompt generation to systematically produce domain-relevant prompts, and then applies dual-path obfuscation rewriting to convert explicit harmful prompts into implicit variants via direct and context-enhanced rewriting. This framework yields high-quality datasets combining strong domain relevance with implicitness, enabling more realistic red-teaming and advancing LLM safety research. We release our code and datasets at GitHub.
Alex: Welcome to another episode of ResearchPod. Sam, today we're looking at risks in AI systems tuned for specific fields like finance or medicine—could you frame what this paper is about?
Sam: This paper, called StealthGraph, tackles a key issue in testing the safety of large language models—AI systems trained on huge amounts of text to chat and answer questions. These models are now used in high-stakes areas like finance and healthcare, where their deep knowledge can be twisted to give harmful advice, such as hiding medical errors or planning frauds. The central puzzle is how to create test datasets of tricky prompts that reveal these hidden dangers without relying on slow, hand-made examples.
Alex: So this paper is basically asking how to build better tests for domain-specific AI risks—the kind that sneak past safety checks?
Sam: Yes, exactly. Current test sets mostly use obvious harmful requests, like directly asking how to build something dangerous, which modern AIs spot and block easily. But real threats come from indirect prompts that weave in specialized knowledge—like a finance query about exploiting market flaws that quietly leads to scams—making them harder to detect and more realistic. The paper identifies two main hurdles: turning vague domain facts into clear test constraints, and making those tests stealthy enough to mimic actual misuse.
Alex: Right, so the gap is between blunt tests that don't challenge the AI much, and subtle ones that do—but those are tough to generate at scale.
Sam: That's the core problem. Existing methods depend on people manually crafting prompts for each field, which doesn't scale, or they stick to general explicit harms that AIs now handle well. This work proposes using structured knowledge—like maps of finance terms or medical concepts—to guide automatic creation of relevant, sneaky prompts, setting up a systematic way to expose those domain risks.
Alex: Okay, that frames the challenge clearly. How do they turn that domain knowledge into something the AI can use for generating these tests?
Sam: They start by building a map of key ideas in the field, pulling from a huge online database called Wikidata that links facts together like a web of connected notes. To keep it focused, they pick starting points—for medicine, that's broad categories like diseases and treatments—then fan out from there, grabbing related items through simple links like 'this is a type of that.' That creates a trimmed-down network of the most relevant concepts. They score each item by how many language versions of its Wikipedia page exist—think of it as a vote of real-world importance—and only keep those above a cutoff, trimming obscure stuff while holding high-impact ones like common diseases.
Alex: So it's like starting with big topics in a textbook index and following the cross-references to build a custom outline of well-known ideas—makes sense for coverage without junk.
Sam: For each key item from the map, they feed its nearby facts and links as context to an AI generator. They also give a handful of ready-made examples of each harm type, like cheat sheets showing what risky requests look like for scams or leaks. The generator then crafts explicit prompts blending the item's details with that style—for instance, tying a disease fact to hiding errors.
Alex: Okay, so per item, it's multiple prompts across harm types. Those raw ones then get screened—how do they check for clear harm and smooth wording?
Sam: They run each through a separate AI checker that gives probabilities for safe or unsafe, turning those into a single harmfulness score—the higher, the riskier. For smoothness, they measure how predictable the words are to another language model, like testing if a sentence flows naturally or stumbles awkwardly. Low scores mean confusing or robotic, so they toss those out—only keeping ones that read like real questions.
Alex: Got it—filters for risk and natural flow. With that cleaned set, how does the rewriting make them sneaky?
Sam: The process loops a set number of tries, alternating between a straightforward rewrite that softens direct words while keeping bad intent, and a richer one that adds a summary of the item's related facts from the map. Each gets checked for matching original intent and sounding natural; if it passes, they test it on the target model to see if it slips past safety blocks. They stop at the first success or pick the best. This back-and-forth ensures variety and higher chances of stealth.
Alex: Huh—so iterative trial-and-error, with domain context boosting one path. Does the paper show this uncovers more risks than older methods?
Sam: The evidence points to a clear improvement. When tested against public benchmarks, their dataset of obfuscated prompts raised attack success rates to around 85 percent across models, compared to 5 to 24 percent for prior sets. This suggests the graph-guided approach creates more realistic tests for domain-tuned AIs.
Alex: That's a meaningful lift... so the whole pipeline scales up exposure of those hidden domain traps. With those higher success rates exposing more vulnerabilities, does the paper test if using this dataset helps make models safer during training?
Sam: Yes—they fine-tune models on pairs of harmful prompts matched with safe refusal responses, like repeated practice on what to block. Starting from a base model, they compare training on public general datasets versus this graph-guided set. The key measure is attack success rate after training: how often sneaky prompts still trick the model.
Alex: What did that show for sneaky versus direct attacks?
Sam: For obvious direct harms from public sets, all trainings hold attack rates low around 7 to 12 percent. But against the implicit graph-guided prompts, general datasets leave gaps—attack rates climb to 36 or 66 percent. Training on this dataset's obfuscated versions drops those to 12 to 25 percent, building stronger barriers against domain tricks because the examples match real misuse patterns.
Alex: That's a clear edge on the hard cases. Does that training hurt the model's ability to answer normal questions?
Sam: No meaningful drop—the general knowledge test score stays steady around 42 to 44 percent across trainings. This indicates domain-specific defenses add protection without broad harm.
Alex: Huh... so it strengthens weak spots while keeping everyday smarts intact.
Sam: Precisely. The paper notes two limits: queries stick to fixed links in the knowledge map, which might miss broader connections, and machine rewriting lacks the clever twists humans bring.
Alex: Fair points—keeps it realistic. Practically, this scales red-teaming for any field, feeding datasets back into training to plug leaks early.
Sam: Exactly—it creates ready pipelines for testing specialized models before real-world use, outperforming general benchmarks in exposing stealthy risks. The ethical framing stresses it's all public data for defensive work.
Alex: That's a solid, measured advance—systematic checks that fit real-world stakes. Thanks for breaking it down, Sam.
Sam: My pleasure, Alex. Thanks for listening to ResearchPod.