Random Quadratic Form with random forcing: Metastable synchronization by noise

This paper investigates the synchronization and clustering properties of the Random Quadratic Form (RQF) on a sphere when perturbed by random Brownian forcing. The RQF without forcing is known to exhibit partial synchronization, where trajectories converge to an anti-polar configuration due to intrinsic symmetries. However, continuous-time machine learning models—such as Neural ODEs and continuous-time transformers—often incorporate both weight matrices and small random biases, which correspond to additive forcing terms in the underlying dynamical system. Understanding how an arbitrarily small perturbation affects the long-term behavior of these systems bridges the gap between idealized symmetric models and realistic neural network initializations.

The author models the system using stochastic differential equations on the unit sphere and analyzes the associated Random Dynamical Systems (RDS) and Markov properties. By distinguishing between the fast attractive dynamics of the non-forced RQF and the slow symmetry-breaking effects of the additive noise, the paper tracks the two-point motion across distinct temporal regimes. The study employs infinitesimal generators, Dynkin's formula, and forward Kolmogorov equations to rigorously establish convergence rates in expectation toward both the transient meta-attractor and the global random attractor.

The findings reveal a two-stage multiscale behavior for the forced RQF. In the first stage, driven by the strong attraction of the symmetric configuration on the time scale $t \sim \log|_\gamma|^{-1}$, particles rapidly cluster into an anti-polar configuration associated with the non-forced RQF meta-attractor. In the second stage, the small forcing breaks the symmetry, causing the two clusters to meet and eventually converge to a single global random attractor. Interestingly, despite this dramatic shift in two-point synchronization, the one-point motion remains a rescaled spherical Brownian motion unaffected by the forcing parameter. These results explain the role of bias and initialization scales in continuous-time machine learning architectures.