Learning-Induced Dynamical Transition in Recurrent Neural Networks

How does the process of learning, rather than just the final trained state, fundamentally reshape the dynamics of a recurrent neural network (RNN)? The study investigates the mechanism by which a network transitions from internally generated chaotic activity to reliable, stable computational behavior during training.

The author develops a non-equilibrium dynamical mean-field theory (DMFT) to track the network's evolution. Unlike static analyses, this framework uses a two-time correlation function to capture the non-stationary nature of the network as synaptic weights evolve. By modeling learning as a slow, feedback-driven process, the study demonstrates how the growing feedback structure acts as an evolving control parameter that deforms the network's effective dynamical landscape.

The study identifies a critical feedback strength and a corresponding learning-rate-dependent critical time that marks the transition from chaos to stability. This transition is characterized by a bifurcation in the DMFT solution, where the potential well governing the network's fluctuations collapses. The theory successfully predicts the time evolution of the network output and shows quantitative agreement with numerical simulations, confirming that stability emerges dynamically through plastic adaptation rather than being an externally imposed constraint.

This work provides a rigorous theoretical framework for understanding learning as a continuous dynamical process. By connecting microscopic plasticity rules to macroscopic changes in dynamical regimes, it offers insights into how biological and artificial neural circuits reorganize their activity to perform specific tasks. It moves the field beyond characterizing only the final state of a trained network, allowing researchers to study the temporal trajectory of learning itself.