Jingkang Yang, Kaiyang Zhou, Yixuan Li, Ziwei Liu
9 min
Machine learning models are typically trained under a closed-world assumption, where training and test data share the same distribution. However, in real-world deployment, models frequently encounter data that deviates from their training, known as out-of-distribution (OOD) samples. This survey addresses the confusion arising from various, often overlapping, research fields—specifically anomaly detection (AD), novelty detection (ND), open set recognition (OSR), outlier detection (OD), and OOD detection itself. By introducing a unified framework, the authors clarify that these topics, while sharing the common goal of identifying unknown samples, differ in their definitions of training and test sets, their label space requirements, and whether they involve transductive or inductive learning.
The survey categorizes existing detection techniques into four primary approaches: classification-based, density-based, distance-based, and reconstruction-based methods. Classification-based methods, which leverage model outputs or confidence estimates, are the most prominent, ranging from simple post-hoc adjustments (e.g., using max softmax or energy scores) to complex training strategies involving outlier exposure or synthetic data generation. The authors highlight the transition from earlier, heuristic-based methods to modern strategies that incorporate more principled uncertainty modeling and leverage large-scale foundation models, such as CLIP.
The authors analyze representative methods using a fair, unified benchmarking setup (OpenOOD), focusing on CIFAR-10 and CIFAR-100 datasets. Their experiments reveal several key findings:
The field faces significant hurdles, including the need for more rigorous, large-scale benchmarks that avoid label leakage between training and OOD datasets. Future research should prioritize 'outlier-free' methods that do not rely on large, pre-collected OOD training data. Additionally, the authors advocate for the development of models that simultaneously achieve robust classification on known distributions and accurate rejection of semantic shifts, moving toward a 'full-spectrum' detection capability.
Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. We then review each of these five areas by summarizing their recent technical developments, with a special focus on OOD detection methodologies. We conclude this survey with open challenges and potential research directions.
Alex: So distinguishing the known digits from each other is not part of that task, even if the method uses their labels.
Sam: Open set recognition adds that requirement. It must detect unfamiliar digit categories while still correctly classifying familiar digits. For ordinary multiclass classification, the survey says open set recognition and OOD detection largely share this goal.
Alex: If they share the goal, why keep both names?
Sam: Their benchmark traditions and permitted methods differ. Open set recognition usually divides one dataset into known and unknown categories. OOD detection typically uses one dataset as known and other datasets as unknown, with no overlapping categories. Open set recognition also discourages additional training data because of its theoretical constraints.
Alex: We have covered novelty detection and recognition with rejection. Where do anomalies and outliers fit?
Sam: Anomaly detection starts with a definition of normality and treats normal inputs as a whole, without requiring their classification. It can target appearance changes or unfamiliar categories. Outlier detection is different again: all observations are available together, and the majority defines what counts as normal.
Alex: Then the same detection technique could appear in several literatures, while the actual task changes underneath it.
Sam: That is why the map matters. The framework is not a new detector. It is a way to identify the problem and transfer useful methods without overlooking differences in requirements.
Alex: Once the task is clear, what choices does the methods review offer? A listener cannot carry an inventory of algorithms.
Sam: Keep a few broad strategies. Classification-based methods use the classifier's outputs or internal features to distinguish known from unknown inputs. An early baseline used the largest predicted class probability. Later methods change the scoring rule, alter internal activations, or modify training.
Alex: Would that require retraining a model that is already deployed?
Sam: Not necessarily. Post-hoc detection means adding detection without changing training. ODIN, for example, adjusts how outputs are scored and slightly perturbs inputs to improve separation. The survey stresses that avoiding retraining can matter when its cost is prohibitive.
Alex: What about modeling what familiar data look like, then rejecting anything sufficiently improbable?
Sam: That is the density-based approach: explicitly estimate the known data distribution and flag low-probability inputs. But the survey reviews a serious complication. Some probabilistic models assign higher likelihood to unknown inputs than to training data. A plausible-looking probability is not automatically a useful rejection score.
Alex: Distance seems like another route: ask whether an input resembles any familiar example.
Sam: Distance-based methods do that in a learned feature space, comparing inputs with class representatives or nearby training examples. Reconstruction methods instead try to recover the input and use differences in reconstruction behavior. These families overlap across the tasks, which is another reason to connect their literatures.
Alex: How much does the survey test rather than organize? I would want to know whether comparisons use the same underlying model.
Sam: Its experiments standardize the network architecture and common training settings. One benchmark uses CIFAR-10, with fifty thousand training images. Its test set contains ten thousand images. The authors separate unfamiliar inputs into near and far groups.
Alex: What makes an unfamiliar input near or far in this comparison?
Sam: Near inputs have unfamiliar categories without an obvious domain change. Far inputs also have an obvious change in appearance or domain. The authors remove overlapping categories from the unknown datasets; about twelve hundred TinyImageNet images are excluded from the CIFAR-10 comparison.
Alex: Otherwise, the benchmark could penalize a model for recognizing a category it was actually trained to recognize.
Sam: That is the evaluation flaw the authors warn about. In their results, the augmentation method PixMix scores ninety-three point one percent on near-unknown detection with CIFAR-10, best among the compared methods there. The metric summarizes separation across decision thresholds; it is not classification accuracy or a deployed rejection rate.
Alex: Does that make augmentation the general recommendation, or is the comparison narrower?
Sam: Narrower. Augmentation methods perform strongly across these benchmarks, and post-hoc methods generally outperform methods requiring training. But the authors explicitly warn that strategies effective on CIFAR may not work as well on the larger ImageNet category space. I would treat these rankings as useful baselines, not deployment guarantees.
Alex: And does strong separation show that a detector preserves recognition when only appearance changes?
Sam: No. These experiments exclude pure covariate-shift detection. The survey also cites evidence that many detectors respond more strongly to appearance changes than category changes. Its proposed direction is to handle changed appearances while still detecting unfamiliar categories.
Alex: So who should read the full survey, and where should they begin?
Sam: Researchers building or evaluating rejection mechanisms should start with the task definitions in Section Two. Then read the experimental comparisons in Section Five, especially the near-versus-far distinction. For robustness research, prioritize the discussion of detection and generalization in Section Six. There are application-specific exceptions, including conservative medical settings, so the intended boundary must be explicit.
Alex: And what should everyone else remember when they hear that a model detects unknown inputs?
Sam: Ask what it rejects, and what it should still recognize. A useful detector must distinguish a new category from a new look.
Alex: That is a boundary worth checking.