IGME: Efficient Chained Method Ensemble for Transferable Semantic Segmentation Attacks

Semantic segmentation models are highly vulnerable to adversarial perturbations, but evaluating these vulnerabilities via transfer attacks is computationally expensive. Traditional ensemble attacks, which aggregate information from multiple surrogate models, are particularly costly for dense prediction tasks because they require repeated forward and backward passes through the source model. This paper investigates whether a single-source approach can achieve competitive transferability without the computational overhead of model ensembles.

The authors introduce IGME (Integrated Gradient Method Ensemble), which reformulates transferable attack composition as a chained computation over differentiable attack components. By placing these components within a single computation graph, the expensive source-model gradient computation is shared rather than repeated. To address the potential update instability caused by this chained composition, the authors incorporate an integrated-gradient-style path-averaged direction as an empirical stabilization heuristic. This allows the attack to leverage multiple gradient directions along the path from a baseline image to the adversarial example, effectively stabilizing the update without requiring multiple surrogate models.

IGME demonstrates a favorable trade-off between transferability and computational efficiency. Experimental results on Pascal VOC and Cityscapes show that IGME achieves competitive transferability compared to single-source baselines while maintaining significantly lower runtime than traditional model-ensemble attacks like SVRE. While model-ensemble methods remain strong, IGME provides a viable, efficient alternative for scenarios where access to only one source model is available. The study confirms that chained component composition, when stabilized by path averaging, can effectively transfer adversarial effects across both CNN- and transformer-based segmentation architectures.