Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

Sparse autoencoders (SAEs) are increasingly used to decompose high-dimensional neural network activations into interpretable features. However, evaluating whether these features truly correspond to human-understandable concepts remains difficult, as existing metrics often rely on structural proxies or qualitative inspection rather than direct semantic validation. This paper addresses the need for a rigorous, intervention-style evaluation framework for vision-based SAEs.

The authors propose a three-part evaluation framework:

1. Fully-Binary Matching Pursuit (FBMP): A greedy, coalition-based algorithm that aligns SAE latents with human-annotated concepts. Unlike one-to-one baselines, FBMP supports many-to-one mappings, allowing it to account for feature splitting where a single concept is distributed across multiple latents.
2. Synthetic Benchmarks (synCUB and synCOCO): The authors constructed two datasets of paired images that differ by exactly one attribute. This allows for controlled, intervention-style testing of whether a latent actually represents a specific visual concept.
3. TAPAScore: A functional metric that measures whether the latents matched to a specific attribute respond selectively and in the expected direction when that attribute is added or removed in the synthetic image pairs.

The study demonstrates that the proposed matching and TAPAScore metrics are the only ones capable of reliably distinguishing between trained SAEs and untrained (random) baselines. The authors find that while increasing the dictionary size (overcompleteness) improves statistical matching scores, it often degrades perturbation alignment, suggesting that larger dictionaries do not necessarily lead to more interpretable features. They conclude that moderate dictionary sizes provide the optimal trade-off for interpretability.