Origins and mitigation of double descent in reduced order modeling

Sparse sensing allows for the reconstruction of high-dimensional systems from a limited number of localized measurements. However, these reconstruction processes often exhibit 'double descent'—a counterintuitive phenomenon where the reconstruction error spikes as the number of sensors increases, before eventually decreasing again. This paper seeks to explain the origins of this instability and provide a framework to predict and mitigate it in reduced order modeling (ROM).

The authors develop a unified 'Data-Noise Averaging' (DNA) theory to model the reconstruction risk. Unlike previous approaches that rely on random matrix theory and generic ensembles, the DNA theory uses the specific singular value decomposition of the training data and the chosen sensor locations. This allows for a closed-form analytical expression of the reconstruction risk curve that accounts for the specific design choices of the sensing setup, such as sensor placement algorithms, noise levels, and regularization strategies. The authors validate this framework using both static Sea Surface Temperature (SST) data and time-integrated partial differential equations (PDEs).

The study identifies that the double descent spike is caused by the catastrophic amplification of 'pathological signals'—specifically, measurement noise and truncated modes—when the inversion matrix becomes ill-conditioned. This typically occurs when the number of sensors is close to the number of latent modes. The DNA theory provides a computationally efficient way to predict these risk curves, achieving a 1000x speedup compared to empirical averaging. Furthermore, the authors demonstrate that this instability can be effectively mitigated by incorporating Tikhonov (ridge) regularization into the reconstruction matrix, which suppresses the resonant amplification of noise and leads to more stable, accurate models.