SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization

Interpreting the internal representations of large language models often relies on sparse autoencoders to extract features, but explaining what those features mean remains a major bottleneck. Conventional bottom-up methods require collecting top-activating text examples across a massive corpus and prompting an external LLM to summarize their shared pattern. This approach is computationally expensive and yields superficial explanations derived from observable model behavior rather than internal representations. The authors introduce SAEVerbalizer, a framework that shifts feature explanation from external behavioral observation to direct internal representation processing.

The framework treats feature explanation as a trainable capability of the language model itself. During prompt prefilling, a feature-agnostic task prompt is fed into the verbalizer, and the sparse autoencoder decoder direction for a specific feature is added directly to the token representations at a designated injection layer using a norm-matched additive operation. Only the Transformer layers downstream of the injection point are fine-tuned using high-quality feature-explanation pairs. This design freezes upstream layers to preserve the representation space while enabling the model to condition its autoregressive generation directly on the injected direction.

Experiments across various model scales and layers demonstrate that the learned verbalization capability generalizes effectively to unseen features. The approach scales with model capacity, achieving higher reference agreement on larger backbones. Furthermore, the framework extends to separately trained sparse autoencoder dictionaries and, with the addition of a lightweight affine adapter, successfully translates decoder directions from completely different source large language models into the verbalizer's representation space without requiring new feature-explanation supervision.