CONTEXT IS KING: HOW IN-CONTEXT SPECIFICATION SHAPES THE GEOMETRY OF CONCEPTS

This paper investigates whether the relational geometry of concepts in large language models (LLMs) is an intrinsic, fixed property stored during pretraining or a flexible structure assembled on-demand from the provided context. While previous research suggests models store fixed world-models (like a circular representation of weekdays), this study tests whether declarative instructions can force a model to reorganize these concepts into arbitrary topologies, such as cycles or branching trees, even when these conflict with strong pretrained knowledge.

The authors employ a series of "conflict" experiments where they provide models with declarative rules that contradict established knowledge (e.g., redefining the order of days of the week). They measure the resulting geometry using Representational Similarity Analysis (RSA) on the model's residual stream activations. To ensure the observed geometry is not merely a correlational artifact, they use activation patching—a causal intervention—to determine if the model actually uses this context-imposed map to generate subsequent tokens. They test this across various model families (Gemma, Qwen) and scales, ranging from small base models to large 31B parameter models.

The study finds that context is the primary driver of concept geometry. When a declarative rule is provided, the model's internal representation shifts to match the imposed structure, effectively suppressing the pretrained prior. This is not just a simple reordering; the model can be commanded to change the topology type itself (e.g., from a cycle to a tree). Causal interventions confirm that these maps are not just probe artifacts but are actively used by the model to compute outputs. Crucially, the authors identify that while even small models can form a rough map, the ability to cleanly dominate the pretrained prior and utilize the map causally is a capability that emerges only at larger scales.