GS-CPE: Unified 6-Degree-of-Freedom Camera Pose Estimation via 3D Gaussian Splatting

How can we achieve precise 6-Degree-of-Freedom (6-DoF) visual localization that balances high accuracy with robustness to viewpoint changes and occlusions, while avoiding the high computational costs of traditional neural rendering methods?

The authors introduce GS-CPE, a two-stage framework that leverages 3D Gaussian Splatting (3DGS) for efficient scene representation and pose refinement.

1. Coarse Estimation: The system begins by retrieving a reference image from a database using NetVLAD. It then performs iterative 2D-3D feature matching between the query image and views rendered from the 3DGS model, solving for the initial pose using PnP-RANSAC.
2. Refinement: The coarse pose is refined through a multi-scale, warping-based optimization. This stage minimizes a photometric loss between the query image and a warped reference view. To ensure stability, the authors incorporate three key mechanisms: adaptive re-linearization (re-rendering the scene when the pose update exceeds a threshold), a visibility-aware mask (to ignore occluded regions), and a multi-scale pyramid approach to improve convergence.

GS-CPE demonstrates state-of-the-art performance across standard indoor (7Scenes) and outdoor (Cambridge Landmarks) benchmarks, as well as real-world datasets like FAST-LIVO2. By combining the global robustness of geometric initialization with the local precision of photometric warping, the framework consistently outperforms existing Absolute Pose Regression (APR), Scene Coordinate Regression (SCR), and Neural Render Pose (NRP) methods. The visibility-aware masking and adaptive re-linearization specifically help the system maintain accuracy even in challenging scenarios with significant appearance gaps or clutter.

This work provides a practical solution for robotics and augmented reality applications where accurate localization is critical but must be achieved efficiently. By utilizing 3DGS, the method bridges the gap between high-fidelity neural rendering and the real-time requirements of autonomous systems, offering a more scalable alternative to computationally heavy NeRF-based approaches.