Sekh Mainul Islam, Pepa Atanasova, Isabelle Augenstein
8 min
Personalizing Large Language Models (LLMs) is essential for user-centric applications, but doing so in dense models is often a black-box process. Because dense LLMs use polysemantic neurons—where one neuron might represent many unrelated concepts—it is difficult to determine which components of the model actually handle specific user preferences. This paper proposes using Mixture-of-Experts (MoE) LLMs to solve this. By routing tokens to specialized experts, MoE architectures naturally reduce polysemanticity, making them more amenable to mechanistic interpretability.
The authors adapt three open-weight MoE LLMs (OLMoE, Qwen3, and Mixtral) for personalization using a two-stage fine-tuning process: first optimizing the routers to select preference-relevant experts, and then fine-tuning only those experts. To interpret these changes, they train sparse autoencoder-based crosscoders on activations from both the base (non-personalized) and personalized models. This allows them to map specific personalized latents—abstract features that drive personalization—to the specific experts that process them. They validate their findings using causal interventions and probing techniques to see if boosting specific experts can steer non-personalized models toward the behavior of their personalized counterparts.
The experiments show that MoE LLMs provide a scalable path for personalization, matching or exceeding dense models while using only about 20% of the parameters per token. Critically, the MoE structure proves significantly more monosemantic, with latents cleanly mapping to small, stable sets of experts. Causal interventions confirm that these localized expert paths are both sufficient to recover personalized behavior and necessary for its execution. This work provides a rigorous foundation for building more transparent, controllable personalized models.
Personalization of Large Language Models (LLMs) is increasingly important for user-centric applications, yet prior methods mainly adapt dense LLMs, whose polysemantic neurons, each encoding multiple unrelated concepts, hinder localizing the components behind personalization, and thus understanding and controlling it. We propose to personalize Mixture-of-Experts (MoE) LLMs, which sparsely route each token to a few specialized experts, reducing polysemanticity and enabling such localization. We adapt three open-weight MoE LLMs on a large-scale personalization dataset and train crosscoders on paired activations of the personalized and non-personalized models to learn interpretable personalized latents, from which we identify the experts encoding distinct personalization features. The personalized MoE LLMs perform competitively with dense LLMs while activating only a fraction of their parameters, exhibit lower polysemanticity than dense LLMs with similar active parameters, and, within the same model family, yield personalized latents that are less shared with the non-personalized model. Causal interventions show that these latents suffice to recover the personalized model's behavior and that boosting their associated experts moves the non-personalized model toward it, revealing distinct routing paths for different personalization features.
Sam: They first train only the routers, leaving the experts unchanged. That identifies experts frequently selected on the preference data. Then they freeze the routers and train only the most selected experts in each layer.
Alex: They separate choosing the computational route from changing what the selected components do. Does that produce useful personalized answers?
Sam: Across three model families, the adapted expert models outperform or match adapted dense models with similar active parameter counts. Active parameters are the ones used for a particular token. Comparisons with dense models of similar total size are also competitive, especially for Qwen and Mixtral.
Alex: That distinction matters: a model can contain many parameters while using relatively few. How small was the active fraction here?
Sam: The paper summarizes it as roughly twenty percent of total parameters per token. But that isn’t a direct measurement of deployment cost or speed. The result is competitive personalization while activating only a fraction of the model.
Alex: Useful performance establishes that the adaptation worked. How do they turn a difference between models into something interpretable?
Sam: Here’s the central tool: a crosscoder. It learns a dictionary of internal features that jointly reconstruct activity from the personalized and unpersonalized models. Those learned features are called latents.
Alex: If a feature helps reconstruct both versions, it may describe something they share. What marks a feature as specifically related to personalization?
Sam: The authors compare how strongly each feature contributes to reconstructing each model. They select features whose contribution strongly favors the personalized model. Across the models, that selects less than a quarter of one percent of the learned features.
Alex: Give me one example of what such a feature represents, rather than just how it was selected.
Sam: In Qwen, one selected feature responds to language indicating the audience’s expertise or background knowledge. Its explanation includes words such as “expert,” “advanced,” and “familiarity.” The authors then identify experts selected more often when that feature fires than when it doesn’t.
Alex: That could identify words in a preference instruction without showing that the model follows the instruction. Where do these features mostly activate?
Sam: That’s an important qualification. In Qwen, roughly four in five selected features fire mainly on the system message. OLMoE shows a similar pattern, while Mixtral has more response-focused features. Much of the map therefore concerns representations of requested preferences, not necessarily their successful execution.
Alex: And the labels themselves come from another language model. How reliable are descriptions like “audience expertise”?
Sam: A language model writes explanations from activating text examples. A scorer then uses those explanations to distinguish activating from non-activating examples. Overall scores are moderate, and stronger activations are easier to explain. These are useful descriptions, not definitive semantic labels.
Alex: Let’s move from descriptions to tests. What makes the evidence more than a correlation between features and routing?
Sam: They intervene in two ways. First, they replace internal activity in the personalized model with activity from its unpersonalized counterpart. Then they add back only the selected personalization features and compare the resulting next-token predictions with the original personalized model.
Alex: The relevant comparison is whether those selected features restore more than an equally sized arbitrary feature set.
Sam: They do better than random features and features shared between the models. That supports their relevance to personalized behavior. But the measured restoration is a reduction in prediction-distribution differences, not proof that the features recover every aspect of personalization.
Alex: What about the experts themselves? A feature could map to an expert without that expert offering a useful control point.
Sam: The second intervention boosts routing toward experts associated with each preference dimension in the unpersonalized model. Its predictions move closer to the personalized model than with random expert steering. That supports a causal role for those routes, although it doesn’t establish a complete mechanism or exclusive ownership of a preference.
Alex: How far should we generalize beyond this controlled preference setting?
Sam: That’s the biggest limitation. Training uses synthetic English preferences stated explicitly in system messages. Evaluation also includes benchmarks requiring preferences inferred from individual interaction histories, but gains are smaller there. Real histories may contain noisier, more entangled signals.
Alex: So there’s evidence beyond the training format, but the strongest story remains explicit group preferences. Are the architecture comparisons clean?
Sam: Not completely. The dense and expert models differ in training data and recipes, not just architecture. The crosscoder also misses some internal activity, and the detailed analysis uses one selected layer per model. I’d read this as a promising localization method, not a comprehensive explanation of personalization.
Alex: Who should read the full document, and where should they start?
Sam: Researchers working on model personalization, interpretability, or expert routing should read it. Start with the causal analysis: compare feature patching with expert steering, and track what each test measures. Then read the limitations before the method for mapping features to experts. For everyone else: personalization can leave identifiable internal routes, but finding a route is not the same as explaining the whole behavior.
Alex: Keep that distinction in mind when you read the results.