The Unseen Hand: Manipulating Model Fairness and SHAP with Targeted Identity Re-Association Attacks

As machine learning models are increasingly deployed in high-stakes societal domains, auditors rely on fairness metrics (like those in AIF360) and explainability tools (like SHAP) to ensure accountability. This paper investigates whether these auditing mechanisms are themselves vulnerable to adversarial manipulation. Specifically, the authors ask if it is possible to create subtle, data-agnostic attacks that can simultaneously 'fix' a model's perceived fairness and hide its reliance on protected features without leaving detectable artifacts.

The authors introduce Targeted Identity Re-Association (TIRA) attacks, a family of algorithms that operate on the model's output scores rather than the input data or model internals. The adversary treats the model as a black box, querying it to obtain prediction scores and then iteratively re-associating identities (e.g., protected group status) with those scores. The authors formalize two specific algorithms:

• Probabilistic Micro-Shuffling (PMiS): Performs localized, adjacent swaps of identities between advantaged and disadvantaged groups based on a swap probability.
• Probabilistic Rank-Shift Micro-Perturbation (PRSMP): Introduces randomized, small-window rank shifts to identity-score associations.

These methods are designed to be more granular and stealthy than prior 'shuffling' attacks, which often left behind detectable patterns.

Empirical testing on the Bangladeshi Diabetes and German Credit datasets across logistic regression and neural network models shows that TIRA attacks are highly effective. The attacks successfully push fairness metrics toward ideal thresholds, demonstrating that a model can be made to appear fair through post-hoc manipulation. Crucially, the authors demonstrate that these attacks effectively 'blind' SHAP, reducing the attribution of protected features to effectively zero. This dual-pronged success—manipulating both fairness metrics and explainability scores—represents a significant improvement in stealth and efficacy over existing adversarial strategies.