REDUCING INFORMATION DEPENDENCY DOES NOT CAUSE TRAINING DATA PRIVACY. ADVERSARIALLY NON-ROBUST FEATURES DO.

This paper investigates the underlying causes of training data leakage in high-resolution image classification models. While the prevailing consensus in machine learning suggests that privacy vulnerabilities arise from excessive information dependency—specifically rote memorization of training data—the authors challenge this view, arguing that it fails to explain why certain models remain vulnerable or robust to Model Inversion Attacks (MIAs).

The authors conduct three key experiments to test the information dependency hypothesis. First, they evaluate whether existing privacy defenses actually reduce standard measures of information dependency (HSIC). Second, they test whether models that perform perfect rote memorization are inherently more vulnerable to reconstruction. Third, they train models on images where over 97% of the pixels are deleted, effectively removing the information that theory suggests should be necessary for reconstruction. Finally, they introduce 'Anti-Adversarial Training' (AT-AT), a training regime that intentionally forces models to learn 'non-robust' features—generalizable but unstable patterns—to mitigate reconstruction risk.

The study reveals that reducing information dependency does not guarantee privacy. The authors find that models can achieve perfect rote memorization without being vulnerable to MIA, and conversely, models trained on nearly empty images (where reconstruction should be theoretically impossible) remain highly susceptible to it. The researchers conclude that privacy under MIA is causally linked to the model's reliance on non-robust features. By intentionally shifting models toward these features, the authors demonstrate that they can significantly reduce reconstruction rates while maintaining high classification accuracy, establishing a new privacy-robustness trade-off.