TooBad: Backdoor Diffusion Models with Ultra-Low Poison Rate and Imperceptible Trigger

Diffusion models (DMs) are increasingly vulnerable to backdoor attacks, where a malicious trigger causes the model to generate a specific target output. Existing attacks face a 'quadrilemma' trade-off: they struggle to balance high attack performance, low poison rates (the percentage of training data manipulated), stealthiness, and low time complexity. The authors ask whether optimizing the trigger itself, rather than using arbitrary patterns, can overcome these trade-offs.

The authors propose TooBad, a framework that optimizes a trigger specifically tailored to the diffusion process. Unlike previous methods that use fixed triggers (like a stop sign) or auxiliary classifiers, TooBad optimizes the trigger by minimizing the distance between the model's output with the trigger and the desired target image during the denoising process. This optimization is constrained to ensure the trigger remains imperceptible (using sparsity and invisibility constraints) and is performed before the actual backdoor injection phase. By injecting this optimized trigger, the model learns the malicious association much faster and with significantly less poisoned data.

TooBad significantly outperforms existing state-of-the-art methods like VillanDiffusion and UIBDiffusion. On the CIFAR-10 dataset, TooBad achieves an Attack Success Rate (ASR) of over 85% with a poison rate of only 0.5%, whereas prior methods typically require 10% or more. At a 5% poison rate, TooBad reaches nearly 100% ASR in just 3-5 training epochs, compared to the 30-50 epochs required by existing methods. Furthermore, the optimized triggers remain stealthy and maintain high model utility, effectively evading current backdoor detection defenses.