Uncertainty quantification via conformal prediction in data assimilation

Research Context and Approach

Quantifying uncertainty is a fundamental challenge in numerical weather prediction, where chaotic dynamics and model imperfections make deterministic forecasts insufficient. While ensemble-based methods, such as the ensemble Kalman filter (EnKF), are the standard for estimating forecast error, they are computationally expensive and often suffer from sampling errors due to limited ensemble sizes. This paper explores Conformal Prediction (CP) as a machine learning-based alternative to provide reliable, uncertainty-aware forecasts in an idealized one-dimensional modified shallow water (MSW) model.

The authors implement a data assimilation framework where a convolutional neural network (CNN) is trained to map unconstrained EnKF analyses to physically constrained analyses (QPEns). They then apply three variants of CP—Standard CP, Normalized CP, and Conformalized Quantile Regression—to generate prediction intervals for the model variables (wind velocity, fluid height, and rain water content). The goal is to produce intervals that contain the true state with a user-specified confidence level, providing a formal guarantee of reliability that heuristic machine learning methods typically lack.

Key Findings

The study demonstrates that CP can effectively complement traditional ensemble-based uncertainty quantification. By using the QPEns analysis as a proxy for the true state, the authors show that CP methods provide statistically valid coverage, ensuring that the true value falls within the predicted interval at the desired confidence level. The results highlight the trade-offs between different CP variants: while Standard CP is simple and robust, Normalized CP and Conformalized Quantile Regression offer improved efficiency by producing sharper (narrower) intervals that adapt to local uncertainty levels. The authors also explore integrating CP-derived uncertainty directly into the data assimilation cycle, showing that CP perturbations can serve as a viable mechanism for representing forecast uncertainty.

As atmospheric models become increasingly complex and computationally demanding, relying solely on large ensembles for uncertainty quantification is becoming unsustainable. This research provides a pathway for integrating machine learning into operational meteorology with formal statistical guarantees. By offering a distribution-free method to quantify uncertainty, CP helps bridge the gap between high-performance machine learning corrections and the rigorous requirements of data assimilation, potentially leading to more reliable weather forecasts and better decision-making in high-impact events.