Huawei Zheng, Xinqi Jiang, Sen Yang, Shouling Ji, Yingcai Wu, Dazhen Deng
7 min
Abstract
Large language models (LLMs) are increasingly applied in specialized domains such as finance and healthcare, where they introduce unique safety risks. Domain-specific datasets of harmful prompts remain scarce and still largely rely on manual construction; public datasets mainly focus on explicit harmful prompts, which modern LLM defenses can often detect and refuse. In contrast, implicit harmful prompts-expressed through indirect domain knowledge-are harder to detect and better reflect real-world threats. We identify two challenges: transforming domain knowledge into actionable constraints and increasing the implicitness of generated harmful prompts. To address them, we propose an end-to-end framework that first performs knowledge-graph-guided harmful prompt generation to systematically produce domain-relevant prompts, and then applies dual-path obfuscation rewriting to convert explicit harmful prompts into implicit variants via direct and context-enhanced rewriting. This framework yields high-quality datasets combining strong domain relevance with implicitness, enabling more realistic red-teaming and advancing LLM safety research. We release our code and datasets at GitHub.
Alex: So it's like starting with big topics in a textbook index and following the cross-references to build a custom outline of well-known ideas—makes sense for coverage without junk.
Sam: For each key item from the map, they feed its nearby facts and links as context to an AI generator. They also give a handful of ready-made examples of each harm type, like cheat sheets showing what risky requests look like for scams or leaks. The generator then crafts explicit prompts blending the item's details with that style—for instance, tying a disease fact to hiding errors.
Alex: Okay, so per item, it's multiple prompts across harm types. Those raw ones then get screened—how do they check for clear harm and smooth wording?
Sam: They run each through a separate AI checker that gives probabilities for safe or unsafe, turning those into a single harmfulness score—the higher, the riskier. For smoothness, they measure how predictable the words are to another language model, like testing if a sentence flows naturally or stumbles awkwardly. Low scores mean confusing or robotic, so they toss those out—only keeping ones that read like real questions.
Alex: Got it—filters for risk and natural flow. With that cleaned set, how does the rewriting make them sneaky?
Sam: The process loops a set number of tries, alternating between a straightforward rewrite that softens direct words while keeping bad intent, and a richer one that adds a summary of the item's related facts from the map. Each gets checked for matching original intent and sounding natural; if it passes, they test it on the target model to see if it slips past safety blocks. They stop at the first success or pick the best. This back-and-forth ensures variety and higher chances of stealth.
Alex: Huh—so iterative trial-and-error, with domain context boosting one path. Does the paper show this uncovers more risks than older methods?
Sam: The evidence points to a clear improvement. When tested against public benchmarks, their dataset of obfuscated prompts raised attack success rates to around 85 percent across models, compared to 5 to 24 percent for prior sets. This suggests the graph-guided approach creates more realistic tests for domain-tuned AIs.
Alex: That's a meaningful lift... so the whole pipeline scales up exposure of those hidden domain traps. With those higher success rates exposing more vulnerabilities, does the paper test if using this dataset helps make models safer during training?
Sam: Yes—they fine-tune models on pairs of harmful prompts matched with safe refusal responses, like repeated practice on what to block. Starting from a base model, they compare training on public general datasets versus this graph-guided set. The key measure is attack success rate after training: how often sneaky prompts still trick the model.
Alex: What did that show for sneaky versus direct attacks?
Sam: For obvious direct harms from public sets, all trainings hold attack rates low around 7 to 12 percent. But against the implicit graph-guided prompts, general datasets leave gaps—attack rates climb to 36 or 66 percent. Training on this dataset's obfuscated versions drops those to 12 to 25 percent, building stronger barriers against domain tricks because the examples match real misuse patterns.
Alex: That's a clear edge on the hard cases. Does that training hurt the model's ability to answer normal questions?
Sam: No meaningful drop—the general knowledge test score stays steady around 42 to 44 percent across trainings. This indicates domain-specific defenses add protection without broad harm.
Alex: Huh... so it strengthens weak spots while keeping everyday smarts intact.
Sam: Precisely. The paper notes two limits: queries stick to fixed links in the knowledge map, which might miss broader connections, and machine rewriting lacks the clever twists humans bring.
Alex: Fair points—keeps it realistic. Practically, this scales red-teaming for any field, feeding datasets back into training to plug leaks early.
Sam: Exactly—it creates ready pipelines for testing specialized models before real-world use, outperforming general benchmarks in exposing stealthy risks. The ethical framing stresses it's all public data for defensive work.
Alex: That's a solid, measured advance—systematic checks that fit real-world stakes. Thanks for breaking it down, Sam.
Sam: My pleasure, Alex. Thanks for listening to ResearchPod.