Efficient Analytic Uncertainty Quantification for Multi-Modal Regression

Standard regression models often assume a unimodal Gaussian distribution for the target variable, which fails when the true data distribution is multi-modal (e.g., disjoint clusters of user engagement). Existing methods that handle multi-modality often lack efficient uncertainty quantification (UQ) or require computationally expensive ensemble methods. This paper asks: can we develop a unified, distribution-agnostic framework that accurately models complex conditional densities while maintaining the computational efficiency of deterministic networks?

The authors propose a modular framework that integrates a distance-aware spectral backbone with a Variational Bayesian Last Layer (VBLL) engine. They introduce two specific paths for handling multi-modal targets:

1. QR-VBLL (Quantile Regression): Models conditional quantiles using an asymmetric Laplace likelihood, which is analytically tractable within the VBI framework.
2. CR-VBLL (Classification Restoration): Discretizes the target space into bins and models the density as a categorical distribution, using a probit approximation for efficient inference.

Both methods allow for the analytic decomposition of aleatoric (data noise) and epistemic (model ignorance) uncertainty in O(1) time, avoiding the need for costly Monte Carlo sampling or multiple forward passes.

The proposed framework effectively resolves the "Ghost Value" pathology—where unimodal models predict a mean in a low-density region between modes—by allowing the model to represent complex, disjoint conditional densities. Empirically, the authors demonstrate that their approach matches or outperforms computationally expensive ensemble baselines on large-scale benchmarks (WeChat, KuaiRec, Uber). Furthermore, the analytic epistemic uncertainty estimation enables highly data-efficient active learning, allowing models to achieve better accuracy with fewer labeled samples.