Tong Zhang, Motasem Alfarra, Carlos Hinojosa, Christos Louizos, Bernard Ghanem
5 min
As text-to-image generative models advance, ensuring their safety against Not-Safe-For-Work (NSFW) content generation and red-teaming adversarial attacks becomes critical. Existing defenses often rely on white-box assumptions such as weight editing, text encoder optimization, or internal feature intervention, which do not scale to proprietary models. While black-box alternatives like language-model-based prompt rewriting offer broader applicability, they frequently fail against what the authors term the benign adversarial problem: prompts that are linguistically safe yet still trigger harmful visual content due to the model's learned data distribution. To bridge this gap, the authors propose DiSCO, a strictly black-box, prompt-level defense module that operates without modifying or accessing the underlying generative model.
DiSCO views safety defense as a distributional alignment problem in the prompt space. Rather than altering the target generator, DiSCO optimizes the input prompt by finding an effective suffix that steers the generation away from unsafe regions and toward safe regions of the model's output distribution. First, the framework constructs model-specific safe and unsafe reference image pools by querying the target model on non-adversarial prompts from the I2P dataset and filtering them via classifier consensus (using NudeNet and Q16). During optimization, candidate prompt suffixes are generated autoregressively using a lightweight language model in a beam search. Each candidate prompt is scored using a distribution-guided contrastive objective calculated in a frozen CLIP embedding space, maximizing similarity to safe reference images while minimizing similarity to unsafe ones.
Evaluated across multiple architectures—including UNet-based models like Stable Diffusion 1.4 and 2.0, and DiT-based models like Flux and Stable Diffusion 3—DiSCO consistently reduces the Attack Success Rate (ASR) across diverse red-teaming attacks such as Ring-A-Bell, UnlearnDiffAtk, MMA-Diffusion, and P4D. When prepended to undefended models or existing defenses (such as SLD-Max, SAFREE, RECE, and ESD), DiSCO significantly drops average ASR while maintaining or improving semantic fidelity and perceptual quality. Because it is architecture-agnostic and acts entirely upstream of the generator, it serves as a versatile plug-and-play safety booster for any text-to-image system.
As text-to-image generative models advance, they raise critical safety concerns, particularly the generation of Not-Safe-For-Work (NSFW) content such as violence and nudity, further exacerbated by red-teaming adversarial attacks. Existing defenses predominantly operate under white-box assumptions, relying on text encoder optimization, weight editing, or inference-time intervention, and fundamentally cannot scale to proprietary models. Black-box alternatives based on LLM prompt rewriting offer broader applicability, yet fail in a critical regime we identify as the \textit{benign adversarial} problem: prompts that are linguistically safe but still trigger harmful generation due to the model's learned data distribution. We propose DiSCO, a zero-shot, strictly black-box defense that operates entirely at the prompt level as a plug-and-play module, requiring no model retraining, fine-tuning, or access to model internals. DiSCO performs distribution-guided suffix expansion via beam search, optimized through contrastive scoring over safe and unsafe image pools generated by the target model itself, with iterative adaptive feedback until safe content is produced. We demonstrate that DiSCO consistently enhances the safety of both undefended and defended models on the I2P benchmark under multiple red-teaming attacks, achieving 37.7% and 25.13% ASR reduction, respectively, while maintaining semantic fidelity and improving image coherence. As a black-box, architecture-agnostic module, DiSCO can be readily applied to any text-to-image system without necessitating any changes to the model itself.
Sam: So it's essentially using the model's own past behavior as a reference?
Alex: Precisely. That's what makes it a black-box approach — it only ever looks at what the model produces, never at the model's internal settings or weights. No retraining required. You can place it in front of a system you don't own or control.
Sam: And once it knows an image is drifting unsafe, how does it fix the prompt?
Alex: It searches for a better ending, one small piece of text at a time. A separate language model — think of it as a writing assistant — proposes candidate additions to the prompt. The system tests each one, checks whether the resulting image moves closer to the safe group, and keeps the best option. It's a bit like editing a sentence by trying different endings until the meaning lands where you want it.
Sam: Does that make the prompt weird? Like, does it add random words that make the image less accurate?
Alex: The paper found the opposite. Safety improved, and the images stayed close to what the original prompt was asking for. The suggestion is that DiSCO isn't just suppressing content — it's guiding the model toward a more faithful reading of the benign intent behind the prompt. The extra words aren't noise; they're a correction.
Sam: What did the tests actually show?
Alex: The authors report a meaningful drop in unsafe image generations across a range of different attack types and different model families. Crucially, that improvement held even when other safety measures were already in place — meaning DiSCO added something on top of existing defenses rather than simply duplicating them.
Sam: What should we be cautious about when reading those results?
Alex: The paper is careful to frame this as an empirical finding, not a guarantee. It was tested across several attack methods and several model setups, which gives the results some breadth. But it's still limited to the specific conditions the authors evaluated. No single method closes every gap, and new attack strategies could emerge that this approach hasn't been tested against.
Sam: So it's a useful tool, but not a complete solution.
Alex: That's a fair way to put it. The practical value is that it's training-free, works without access to the model's internals, and can be placed in front of systems you don't control. The broader point the paper makes is that safety doesn't always require changing the generator — sometimes you can reduce harm by steering the request itself more carefully. DiSCO is a concrete example of that idea working in practice.
Sam: That feels like a meaningful contribution, especially for anyone deploying these systems in the real world.
Alex: It is. It won't be the last word on image safety, but it offers a workable, careful approach to reducing harm from the outside while keeping the prompt's original meaning largely intact. Thanks for listening to ResearchPod.