Explainable AI in Speaker Recognition - Making Latent Representations Understandable

This paper investigates the internal organization of neural networks trained for speaker recognition. While previous research has often treated network representations as flat, independent clusters, this study explores the hypothesis that these representations possess a deeper, hierarchical structure. By applying hierarchical clustering algorithms to the latent space of a pre-trained speaker recognition network, the authors aim to make the network's decision-making process more transparent and interpretable.

The researchers employ two established hierarchical clustering algorithms: Single-Linkage Clustering (SLINK) and Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN). These algorithms are used to uncover hierarchical relationships within the high-dimensional representation space. To interpret these findings, the authors introduce Hierarchical Cluster-Class Matching (HCCM), a method that maps hierarchical clusters to specific semantic attributes such as gender, nationality, or combinations thereof (e.g., 'female and Ireland').

A significant innovation in this work is the introduction of the Liebig score (L-score). Unlike the conventional F-score, which uses a harmonic mean of precision and recall, the L-score is based on Liebig’s law of the minimum. It identifies the most restrictive factor—either precision or recall—as the primary metric for evaluating the quality of a cluster-class match. This provides a more intuitive and interpretable diagnostic tool for understanding why a specific cluster may or may not align with a target semantic class.

By moving beyond flat clustering and providing a framework for semantic interpretation, this research offers a pathway to 'opening the black box' of speaker recognition systems. The ability to visualize and label hierarchical clusters allows researchers to verify whether a network is learning task-relevant features or relying on spurious correlations, ultimately contributing to more robust and explainable AI systems in speech processing.