Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

Topic models typically define topics as probability distributions over words, which often leads to top-ranked words that fail to capture the nuanced semantic meaning of a topic. Researchers frequently struggle to assign human-readable labels to these word-based distributions. This paper introduces MonoTM, a framework that leverages sparse autoencoders (SAEs) to extract interpretable, monosemantic features from dense document embeddings. The authors hypothesize that the features best suited for statistical document-topic mixture estimation are not necessarily the same features that are easiest for humans to interpret. To address this, MonoTM separates these two roles: it estimates document-topic mixtures using the full SAE feature set and then learns topic-feature distributions over a separate, validated vocabulary of interpretable features.

The authors demonstrate that SAE configurations significantly impact both interpretability and statistical performance. Specifically, low-to-moderate per-document activity levels in SAEs consistently yield the highest fraction of interpretable features. However, these same configurations are not always optimal for document-topic mixture estimation, which favors a balanced scaling where dictionary capacity and per-document activity increase together. By decoupling these roles, MonoTM achieves competitive document-topic mixture estimation while providing more meaningful, human-readable topic descriptors than traditional word-based methods. The authors also audit the features excluded from the final descriptor set, finding that these features typically represent auxiliary formatting or register cues rather than missing core topic semantics.

MonoTM provides a more robust and interpretable alternative to traditional topic modeling, particularly for social science and digital humanities research where traceability and semantic clarity are paramount. By moving beyond word-level descriptors to corpus-grounded semantic units, the framework allows researchers to better bridge the gap between coarse corpus-level themes and individual document content. Furthermore, the shared semantic-feature space enables downstream analyses, such as inspecting how topics relate to one another through shared semantic attributes rather than mere lexical overlap.