Explainable AI in Speaker Recognition – Attention Map Visualisation and Evaluation

Explainable AI (XAI) seeks to make the decision-making processes of neural networks transparent. In speaker recognition, where models classify audio utterances by speaker identity, researchers often use attention maps to visualize which parts of an input (such as a spectrogram) the model prioritizes. While various methods exist to generate these maps, there is a lack of rigorous, standardized evaluation to determine which visualization method most accurately reflects the model's internal logic. This paper addresses this gap by systematically reviewing existing evaluation techniques and proposing an improved methodology.

The authors focus on the Randomised Input Sampling for Explanation (RISE) evaluation framework. This approach evaluates an attention map by progressively masking the input regions it highlights and measuring the resulting drop in model performance; a sharper decline suggests the map correctly identified the most critical information. The authors identify two primary limitations in the original RISE-eval: the evaluation results can become indistinguishable across different maps, and the process is often confounded by factors unrelated to the actual quality of the explanation. To solve this, they propose the Modified RISE-eval, which introduces targeted improvements to the masking and evaluation pipeline to provide a more robust assessment.

The study applies the Modified RISE-eval to two popular Class Activation Map (CAM) variants: GradCAM and LayerCAM. By testing these on a speaker recognition network trained on the VoxCeleb dataset, the authors demonstrate that these methods are not universally superior to one another. Instead, their effectiveness depends on the depth of the network layer being analyzed. GradCAM tends to produce more accurate attention maps when utilizing activations from the deepest layers of the network, whereas LayerCAM provides better insights when extracting information from shallower layers.