Sparse Autoencoders for Interpretable Out-of-Distribution Detection

Research Question and Approach

Machine learning models often struggle to identify out-of-distribution (OOD) inputs, frequently producing overconfident, incorrect predictions. Traditional OOD detection methods typically focus on the final output layer (logits), which often discards valuable hierarchical information captured in earlier layers. This paper proposes SAID, a method that trains sparse autoencoders (SAEs) on intermediate activations of a frozen pre-trained model. By decomposing these activations into sparse, concept-like features, the authors create a more discriminative and interpretable signal for detecting both semantic and covariate distribution shifts.

SAID consistently outperforms standard post-hoc OOD detection baselines across both vision transformer (ViT) and CNN architectures. The authors demonstrate that intermediate layers contain discriminative information that is often compressed or lost in the final layer, particularly for covariate shifts like image corruptions. By aggregating similarity scores across multiple layers, SAID achieves state-of-the-art performance on standard benchmarks. Furthermore, the authors use a vision-language model to annotate the sparse latent features with human-readable labels, providing a diagnostic tool that reveals how OOD inputs fail to activate the semantically consistent concepts expected of in-distribution data.

This work highlights that the "black box" nature of deep neural networks can be partially mitigated by inspecting intermediate representations. By shifting the focus from final-layer confidence to the internal concept space, researchers can better understand why a model fails on specific OOD inputs. This approach not only improves safety through more reliable OOD detection but also offers a pathway toward more interpretable AI systems that can explain their own uncertainty.