Many-body Tipping Dynamics of ChatGPT-like AIs

Why do ChatGPT-like AI models, despite their diverse architectures, training data, and safety guardrails, consistently exhibit 'tipping' behavior—suddenly switching to undesirable, repetitive, or harmful content? The authors investigate whether this phenomenon is an inherently unpredictable stochastic event or a foreseeable consequence of the model's underlying mathematical structure.

The researchers model the LLM as a finite-layer system where tokens act as high-dimensional spins. They treat the attention mechanism as a learned two-body interaction that transports these spins through the model's layers. By analyzing the residual stream as a dynamical system, they identify 'output basins'—regions in the vector space representing different semantic meanings. Tipping is modeled as a first-passage process where the model's internal state crosses a boundary (a 'wall') between these basins. They derive a closed-form threshold for this tipping and validate it across multiple model families, including GPT-2, Pythia, and OPT, without requiring model-specific tuning.

The study reveals that tipping is a deterministic core mechanism inherent to the finite-depth architecture of transformer models. The 'attention disorder'—a measure of how attention weights are distributed across tokens—controls the transport of the model's internal state. When this state crosses the boundary between a 'desirable' and 'undesirable' basin, the model tips. The authors show that this effect scales from individual tokens to entire phrases and sentences. Because this process is governed by the geometry of the residual stream and the layer-dependent attention dynamics, it represents a foreseeable engineering risk rather than an unpredictable failure.