The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attacks

Input Diversity (DI) is a standard technique in adversarial attacks that applies random resizing and padding to inputs to improve transferability. The authors investigate whether this technique is universally beneficial or if its effectiveness depends on the training regime of the surrogate model. To test this, they compare the performance of DI-enhanced attacks across standard and robustly trained models (e.g., PGD-AT) on ImageNet and CIFAR-10. They use a controlled robustness-strength sweep, varying the training budget (epsilon) while keeping the architecture constant, to observe how the DI effect evolves as a model becomes more robust.

The study identifies the Scissors Effect: as the probability of applying DI increases, the transfer success rate (ASR) for standard models rises, while the ASR for robust models declines. This divergence creates a pattern resembling the blades of a pair of scissors. On ImageNet, blind application of DI costs robust surrogates an average of 10.3% in attack success. The authors demonstrate that this is not a property of a single architecture but holds across CNNs, ViTs, Swin, and ConvNeXt models. Furthermore, they show that the harm is not binary; it begins to manifest even in models with very low levels of robustness.

By decomposing DI into resize and translation components, the authors find that approximately 67% of the performance loss is attributable to the resize operation. They explain this through Local Gradient Consistency (LGC), a metric that measures how stable a model's gradients are under input-space perturbations. Standard models exhibit noisy, high-frequency gradients where DI helps by averaging out variance. In contrast, robust models possess smoother, low-frequency gradients where DI's resize operation distorts useful, aligned gradient directions. Based on this, they propose a training-free guardrail called Consistency-Guided Diversity Input (CG-DI), which disables DI when LGC indicates a robust-like gradient regime, effectively preserving attack strength.