SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization | Anyuan Zhuo et al. | ResearchPod

Transcript: SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization

Sam: An expert chosen by a language model may contribute little to the feature you're studying. This study finds recognizable specialties by measuring what experts contribute, rather than just when they're selected. In the model examined, Java programming and front-end web development belong to different experts.

Alex: Those sound like useful distinctions. Is this a new way to build models, or a way to understand models already trained?

Sam: It's a method paper and preprint by Anyuan Zhuo and colleagues at ByteDance Seed and Shanghai University of Finance and Economics. They analyze an existing language model, Qwen3-30B-A3B.

Alex: So the audience is researchers trying to understand how a model distributes its work, not just whether its answers are good.

Sam: Yes, especially people studying model internals and expert routing. It changes what they can measure about that division of labor. We'll unpack the contribution measure, the evidence for fine-grained specialties, and where the interpretation remains uncertain.

Alex: Let's establish what an expert means here. Are these separate models assigned different subjects?

Sam: Not assigned subjects. A mixture-of-experts model routes each token to a subset of internal networks, called experts. A router selects them and weights their outputs. In this model, each expert layer contains 128 experts, but only eight are selected for a token.

Alex: If the router already tells you which experts it chooses, why isn't that enough to identify their specialties?

Sam: Previous analyses often counted expert selections on collections labeled as code, mathematics, or other domains. Some reported domain specialization; others emphasized surface properties like token identity. But those counts impose the analyst's categories and measure selection, not contribution.

Alex: And a mathematics document can contain ordinary narrative. Labeling every token as mathematics could hide the actual distinction.

Sam: That's one of the authors' objections. Another is resolution: “code” may be too broad to explain how a large collection of experts divides its work. The categories might conceal specialization rather than reveal its absence.

Alex: What replaces those categories? You still need some units for describing what each expert does.

Sam: The authors train a sparse autoencoder, a tool that learns features while reconstructing the layer's output. It represents each token's output using a small selection from a much larger feature collection. Those learned features are called latents.

SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization

Anyuan Zhuo, Shaohua Zhang, Yuan Lin, Pinyan Lu

Nov 5, 2026·7:34·arxiv·Original Paper

Mixture-of-Experts (MoE)Sparse Autoencoders (SAE)Expert SpecializationFeature AttributionSparse AutoEncoder (SAE)Latent Decomposition

Listen

8 min

Summary

Research Question

Mixture-of-Experts (MoE) architectures improve scalability by routing tokens to a subset of specialized experts. However, it remains unclear whether these experts truly specialize in human-understandable domains, as previous methods relied on coarse, analyst-defined taxonomy and simple token-counting heuristics. This paper investigates the actual functional specialization of experts by analyzing their contributions in the feature space of the model's own internal representations.

Methodological Approach

The authors train sparse autoencoders (SAEs) on the outputs of MoE layers. Because the MoE layer output is a router-weighted sum of expert outputs and the SAE encoder is affine, the activation of any latent feature can be decomposed exactly into contributions from individual experts. By aggregating these contributions over a large corpus, the researchers construct a dominance graph that maps which experts principally drive the activation of specific latents. They then use an automated interpretation pipeline (GPT-5.5) to label these sets of dominated latents, revealing the specific, fine-grained tasks for which each expert is responsible.

Main Findings

The analysis reveals that expert contributions are highly concentrated: most latent features are driven by only one or two experts. When these dominant sets are interpreted, it becomes clear that experts specialize well below the level of broad domains. For example, within the code domain, the model distinguishes between experts specialized in Java, front-end web development, or specific programming algorithms. The experts are not mere generalists; they exhibit meaningful, interpretable, and distinct roles. Furthermore, the allocation of these experts across domains is non-uniform, with more experts assigned to categories that occupy larger portions of the training data.

Why It Matters

This study moves beyond anecdotal or coarse-grained observations to provide a rigorous, data-driven understanding of how MoE models organize knowledge. By mapping expert roles through their feature-space contributions rather than external labels, the authors provide a framework for future research in expert pruning, targeted model editing, and understanding the relationship between pretraining data composition and architectural specialization.

Key Terms and Definitions

Sparse Autoencoder (SAE) — A neural network component trained to reconstruct model activations using a large, sparse set of latent features that are more monosemantic than individual neurons.
Dominant Set — The collection of latent features that an individual expert primarily drives, identified by measuring the expert's significant contribution to those latents' activations.
Contribution Profile — A quantitative metric representing how much an expert's output contributes to the activation of specific internal features, normalized by the expert's router weight.
Dominance Graph — A bipartite graph linking experts to the latent features they dominate, used to characterize the division of labor within an MoE layer.
Polysemanticity — The tendency of individual neurons in a standard neural network to encode multiple, often unrelated, concepts, which the SAE helps resolve into distinct, interpretable features.

Abstract

Mixture-of-Experts (MoE) models route each token to a few of many experts, yet what an individual expert is responsible for remains unsettled. Existing analyses count how often an expert is selected on domain-labeled corpora, an instrument that fixes the taxonomy in advance, stays coarse, and records selection rather than contribution. We instead study specialization in feature space by training a sparse autoencoder (SAE) on the output of an MoE layer. Since this output is a router-weighted sum of expert outputs and the SAE encoder is affine, the pre-activation of every latent decomposes exactly over the experts. Aggregated over a corpus, this gives each expert a dominant set of latents whose activation it principally drives. On layers 4, 30 and 46 of Qwen3-30B-A3B, contribution is far more concentrated than gating, most latents have only one or two dominators, and the overlap between dominant sets concentrates on a few expert pairs. Auto-interpreting these sets reveals a division of labor below coarse domains, with distinct experts for Java and front-end web code, or for geometry and for arithmetic and polynomials, while the coarse domains themselves take very unequal numbers of experts. Our SAEs and code are available at https://github.com/Bytedance-Seed/SAE-Interprets-MoE.

SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization: Research Paper Podcast