Decision Making Needs Uncertainty Quantification

The Decision-Theoretic Framework

This paper establishes a unified framework for understanding how agents should represent uncertainty to act optimally. The core problem involves an agent observing a variable $o$ and selecting an action $a$ to maximize utility $u(s,a)$, where the state $s$ is hidden. The paper demonstrates that the 'best' way to represent uncertainty is not universal; it is strictly dictated by the agent's objective (risk-neutral vs. risk-averse) and the extent of their knowledge about the environment.

Known Environments: Posteriors vs. Prediction Sets

When the environment distribution is known, the optimal strategy depends on the agent's attitude toward risk. A risk-neutral agent, aiming to maximize average utility, finds that the posterior distribution $p(s|o)$ is a sufficient statistic for decision-making. In contrast, a risk-averse agent—who seeks to guarantee a minimum level of utility—does not need the full posterior. Instead, they can achieve optimal performance using a prediction set $C_{\alpha}(o)$, which contains the true state with a specified probability. By applying a max-min decision rule over this set, the agent can certify a 'value-at-risk,' effectively trading off potential gains for protection against worst-case outcomes.

Unknown Environments: Calibration and Robustness

When the environment is unknown, the agent must address epistemic uncertainty using data. The paper identifies three primary approaches:

1. Fixed Predictors: If a predictor is provided, the agent can act optimally on average if the predictor is 'distribution calibrated'—meaning the predicted probabilities match the true conditional frequencies of the states.
2. Non-parametric Approaches: Using a dataset, the agent can construct 'credal sets' (a set of plausible distributions) and employ distributionally robust optimization to ensure performance guarantees despite finite data.
3. Parametric Approaches: By assuming a model structure, the agent can use Bayesian inference over parameters to derive a predictive distribution, which accounts for uncertainty in the model itself.