Nipping the Butterfly Effect in the Bud: Self-Output Fine-Tuning for Autoregressive Weather Prediction

Deep Learning Weather Prediction (DLWP) models have become the standard for medium-range forecasting, often outperforming traditional numerical methods in efficiency. However, these models rely on autoregressive pipelines—where the model's output at one time step becomes the input for the next. This approach is prone to "error accumulation," where small inaccuracies in early predictions propagate and amplify, leading to rapid performance degradation over long horizons. This phenomenon is analogous to the butterfly effect, where minor initial deviations lead to vastly different long-term outcomes.

The authors investigate why this error growth occurs, moving beyond the assumption that it is merely a result of task difficulty. Through theoretical derivation and empirical analysis, they demonstrate that the autoregressive process creates a vicious feedback loop. Even at the very first inference step, the model's output deviates from the ground truth distribution. While these differences are often imperceptible to human observers, they are significant enough that a simple classifier can distinguish model predictions from real weather data with near-perfect accuracy. This shift forces the model to perform subsequent predictions on out-of-distribution (OOD) data, which triggers further errors and accelerates the divergence of the forecast.

To break this cycle, the authors introduce Self-Output Fine-Tuning (SOFT). Instead of using complex, computationally expensive methods like multi-step rollout training or replay buffers, SOFT focuses on the source of the problem: the first step. By fine-tuning the model on its own one-step predictions, the framework forces the model to adapt to the specific biases and artifacts present in its own output. This effectively turns OOD scenarios into pseudo-in-distribution data, allowing the model to remain stable even when it encounters its own imperfect predictions. Experiments show that this simple, plug-and-play modification achieves state-of-the-art performance on long-horizon forecasting tasks while substantially reducing distributional discrepancy.