How Language Models Organize and Structure Moral Knowledge

This paper investigates how large language models (LLMs) organize moral knowledge beyond simple binary detection. The author moves past the "moral vs. neutral" paradigm by training six independent linear probes—one for each category in Moral Foundations Theory (MFT): care/harm, fairness/cheating, liberty/oppression, loyalty/betrayal, authority/subversion, and sanctity/degradation. By extracting the weight vectors of these probes, the study maps the geometric relationships between these moral foundations within the model's high-dimensional representation space.

The research identifies three potential geometric modes: collapse (all foundations merge into one), isolation (foundations are unrelated), and integration (foundations are distinct but share a common structure). The findings consistently point to integration. The foundation directions are separated, spanning a near-maximal number of independent dimensions, yet they share a positive common component. This shared component is significantly more pronounced than in a matched non-moral concept battery, suggesting it is a specifically moral signature rather than a generic artifact of the probing process. This geometric structure is stable across different architectures (dense vs. mixture-of-experts) and scales (1B to 7B parameters).

The study extends this geometric analysis to moral dilemmas—scenarios where two foundations conflict. The results show that dilemma representations are partially compositional: the model's representation of a dilemma significantly overlaps with the subspace spanned by its two component foundations. Crucially, the model maintains a balanced loading between these components, suggesting it represents the tension of the moral conflict itself rather than a pre-resolved judgment. A large residual variance remains, which likely encodes conflict-specific features like trade-off framing.