Latent States in Neural Networks: Recovering the Temporal Structure of Drifting Data from Model Weights

This paper investigates whether artificial neural networks, when trained on a continuous stream of drifting data, spontaneously organize their internal representations into discrete, interpretable regimes. Drawing inspiration from cognitive neuroscience—where human experience is parsed into discrete events—the author asks if a hidden Markov model (HMM) can recover these 'latent states' by analyzing the chronological trajectory of model weights.

The researcher trained independent classifiers on consecutive temporal windows of two datasets known for temporal drift: Fakeddit (multimodal misinformation detection) and Yelp (sentiment analysis). After aligning the weights of these models across different random seeds to account for permutation symmetry, the author applied dimensionality reduction (PCA) and fit an HMM to the resulting weight sequences. The study then evaluated whether these recovered states correspond to functional performance, specifically testing if models generalize better when training and testing data fall within the same latent state compared to when they cross a state boundary.

The study demonstrates that latent states recovered solely from model weights effectively partition the timeline into coherent phases. Models consistently exhibit a 'within-state transfer advantage,' where generalization performance is higher for data sharing the same latent state. This advantage persists even after controlling for temporal proximity and class distribution shifts, indicating that the latent states capture structural information about the data stream that goes beyond simple label changes. While the effect is robust, it is more pronounced in Fakeddit than in Yelp, likely because the latter exhibits a more stable label distribution over time.

This work provides a novel, post-hoc interpretation method for understanding how neural networks adapt to non-stationary environments. By showing that model weights encode the temporal structure of the data, the paper bridges the gap between machine learning dynamics and cognitive theories of event segmentation, offering a way to identify when a model's knowledge has become obsolete due to environmental drift.