OOD Detection for EEG-based Machine Learning in High-Risk Environments

Machine learning models for electroencephalography (EEG) analysis hold significant promise for clinical applications, yet they remain vulnerable to distribution shifts that can cause catastrophic, overconfident prediction failures. This paper investigates how to reliably detect out-of-distribution (OOD) EEG data and examines the practical impact of these detection methods on downstream clinical tasks. To bridge the gap between abstract algorithmic evaluation and real-world utility, the authors introduce a comprehensive evaluation framework for EEG OOD detection.

The study establishes a controlled perturbation suite to simulate realistic distribution shifts—such as changes in sampling rate, channel shuffling, re-referencing, and low-pass or high-pass filtering—with adjustable severity levels. Using two large clinical EEG datasets (including the TUAB corpus), the authors benchmark a wide variety of OOD detection paradigms. These include discriminative-model-based approaches (such as Maximum Softmax Probability, Energy scores, ODIN, and Activation Shaping) and generative-model-based approaches (such as Log-likelihood, Typicality, Density of State Estimation, and Signal in the Noise).

The empirical results reveal a crucial conceptual distinction that is frequently conflated in prior literature: true out-of-distribution detection versus model uncertainty estimation. Discriminative models (trained in a supervised manner) are largely incapable of identifying OOD inputs when evaluated on their activation or softmax outputs alone. In contrast, generative models (trained via flow matching on unsupervised EEG data) successfully detect unfamiliar data, with performance scaling up as perturbation severity increases.

However, when evaluated on their downstream clinical impact—where errors stem from both OOD samples and genuinely difficult in-distribution cases—both paradigms prove useful. While generative models excel at recognizing unusual data distributions, discriminative models serve as highly effective proxies for model uncertainty. The authors demonstrate that combining these complementary signals creates a robust safety net for deploying EEG machine learning models in high-risk environments.