Structure Before Collapse: Transient semantic geometry in next-token prediction

Neural Collapse (NC) theory posits that when models are trained on balanced one-hot classification tasks, their representations inevitably collapse into a symmetric geometry known as a Simplex Equiangular Tight Frame (ETF). In this state, representations are maximally separated based solely on their output labels, effectively erasing any latent semantic relationships between inputs. This creates a theoretical tension: language models are trained on next-token prediction—a task that is effectively one-hot for long contexts—yet they demonstrably learn rich, structured semantic features. This paper investigates how gradient descent recovers this semantic structure despite the one-hot nature of the supervision.

The authors introduce a series of synthetic, controlled languages where inputs possess known latent semantic factors (e.g., color and shape) but are mapped to distinct, one-hot output labels. By using Representational Similarity Analysis (RSA), the researchers track the evolution of the model's internal geometry throughout training. They compare the empirical Gram matrix of the model's embeddings against two theoretical benchmarks: a 'Semantic RSM' that reflects shared latent attributes and an 'ETF-RSM' that reflects the label-driven collapse predicted by NC theory.

The experiments reveal a distinct training trajectory. Early in the training process, the model's representations exhibit a strong correlation with the latent semantic structure of the data, clustering inputs by shared attributes despite the lack of explicit supervision. However, this semantic organization is not the terminal state. As training progresses and the model reaches higher levels of optimization, the semantic geometry weakens and eventually gives way to the symmetric ETF geometry. This confirms that semantic structure is a transient phase that emerges before the model fully succumbs to the label-driven collapse predicted by standard NC theory.