Detecting Pose Estimation Failures via Keypoint Self-Consistency

6D object pose estimation typically involves two stages: detecting 2D keypoints in an image and then solving a Perspective-n-Point (PnP) problem to recover the object's rotation and translation. A major limitation of current methods is that they often predict keypoints independently, ignoring the rigid geometric relationships inherent to 3D objects. This leads to pose estimation failures when individual keypoint predictions are inaccurate. The author investigates whether these failures can be detected by analyzing the spatial consistency of keypoints rather than relying on complex uncertainty quantification models.

The proposed framework, Meta Pose, constructs a set of hand-crafted geometric features that capture the self-consistency of keypoint predictions. These features include pairwise distances between keypoints, reprojection consistency (comparing predicted keypoints to those reprojected from the estimated pose), and render/mask consistency (comparing predictions against rendered views and segmentation masks). A simple logistic regression classifier is then trained on these features to predict whether a pose estimate is correct or incorrect.

Despite its simplicity, the Meta Pose framework consistently outperforms established baselines, such as conformal keypoint prediction, on the challenging LINEMOD Occluded dataset. The study demonstrates that geometric self-consistency is a powerful signal for reliability. Furthermore, the author shows that a subset of these features—specifically those that do not require rendering—can achieve performance comparable to the full feature set, significantly reducing computational overhead while maintaining high accuracy in failure detection.

This work provides a practical, efficient, and interpretable way to improve the robustness of 6D pose estimation pipelines. By enabling reliable failure detection without the need for additional sensors or computationally expensive deep learning components, this approach allows downstream tasks to safely ignore or mitigate unreliable pose estimates, thereby increasing the overall reliability of robotic vision systems.