Discovering Millions of Interpretable Features with Sparse Autoencoders

Research Overview

Sparse autoencoders (SAEs) are a primary tool in mechanistic interpretability for decomposing the dense, superposed representations of large language models (LLMs) into sparse, interpretable features. While SAEs have been successfully applied to several model families, the computational cost of training them has limited the availability of open-source resources. This paper introduces Qwen3-Instruct SAE, a suite of SAEs trained on the Qwen3 instruction-tuned model family (1.7B, 4B, and 8B parameters).

The researchers trained layer-wise SAEs at three distinct activation sites: residual streams, MLP outputs, and attention outputs. They utilized JumpReLU SAEs, which allow for adaptive sparsity patterns across tokens. The models were trained on the FineWeb-Edu dataset and evaluated using both activation-level reconstruction metrics (Fraction of Variance Explained) and model-level recovery metrics (Delta LM loss). To demonstrate practical utility, the authors performed a case study on refusal-steering, testing whether activating specific SAE features could force the model to refuse user requests.

Findings

The evaluation revealed distinct sparsity-fidelity trade-offs across different model components. SAEs trained on residual streams and MLP outputs generally recovered model performance better than those trained on attention outputs, likely because the former encode semantic content more directly. The authors also observed a non-monotonic layer-wise pattern where reconstruction performance dipped in middle layers (2–4) before improving in deeper layers, suggesting that transformer layers serve different computational roles across their depth. Finally, the refusal-steering case study confirmed that selected SAE features can causally influence model behavior, providing a foundation for future safety and interpretability research.