From Keypoints to Predictive Distributions: Post-Hoc Uncertainty for YOLO-Pose Models

Standard YOLO-Pose models are highly efficient for real-time keypoint localization but lack a mechanism to quantify spatial uncertainty for individual keypoints. This paper addresses this limitation by proposing a post-hoc probabilistic extension that augments a frozen, pre-trained YOLO-Pose model. The authors introduce additional lightweight heads that predict a 2x2 dispersion matrix for each keypoint, effectively turning point predictions into bivariate predictive distributions. To ensure these estimates are reliable, the authors employ an importance-weighted negative log-likelihood loss that accounts for differences between training-time label assignment and inference-time post-processing, as well as variations in object scale.

The authors propose two distinct calibration schemes to map the predicted dispersion matrices to meaningful uncertainty estimates: Gaussian calibration, which is ideal for broad compatibility with existing covariance-based downstream tasks, and Student-t calibration, which better captures the heavy-tailed nature of real-world localization residuals. To evaluate these models, the authors introduce a comprehensive protocol that includes Average Keypoint Precision (AKP) to assess reliability ranking, alongside diagnostics for coverage, variance calibration, and quantile-quantile analysis.

Experiments on the COCO dataset demonstrate that the proposed method provides effective keypoint-level reliability rankings. The Student-t calibration is shown to be superior in capturing the empirical distribution of localization errors. Furthermore, the authors demonstrate that these uncertainty estimates can be used for principled keypoint pruning—removing unreliable predictions before they reach downstream processes. A key application highlighted is vision-based aircraft landing, where the calibrated covariances provide the necessary measurement uncertainty for robust aircraft position estimation and sensor fusion.