DREAMSAT-POSE: SPACECRAFT POSE ESTIMATION FROM SINGLE-VIEW 3D RECONSTRUCTIONS AND LEARNED 2D-3D FEATURE MATCHING

Autonomous space operations, such as docking and inspection, require precise 6-DoF pose estimation of uncooperative, unknown targets. Traditional methods often rely on pre-existing CAD models or multi-view depth data, which are frequently unavailable in real-world scenarios. This paper addresses the challenge of estimating the pose of an unknown spacecraft using only a single RGB image, effectively combining generative 3D reconstruction with learned feature matching.

The authors propose DreamSat-Pose, a four-stage pipeline. First, it uses a generative model (Hunyuan-3D-2.0) to reconstruct a 3D shape from a single input image. Second, it extracts 2D visual features using a frozen DINOv3 vision transformer and 3D geometric features from the reconstructed point cloud using a trainable Dynamic Graph CNN (DGCNN). Third, a dual-stream transformer matcher refines these features through self- and cross-attention, producing soft correspondences between the 2D image and 3D model. Finally, these correspondences are fed into a Perspective-n-Point (PnP) solver to recover the 6-DoF pose. The system is trained using a supervised geometric objective that includes soft correspondence, expected-XYZ, and reprojection losses.

DreamSat-Pose demonstrates strong generalization to unseen spacecraft models within the SPE3R dataset. By leveraging reconstructed geometry as a prior, the system achieves a mean pointing error of 0.157 degrees. The results suggest that generative 3D proxies can effectively substitute for a-priori CAD models, allowing for robust pose estimation in scenarios where traditional model-based approaches are infeasible.

This framework bridges the gap between generative AI and classical robotics. By enabling pose estimation without requiring prior knowledge of the target's geometry or multi-view sensing, it significantly expands the operational capabilities of autonomous spacecraft. It provides a viable path for performing complex proximity operations on uncooperative debris or unknown satellites using standard monocular cameras.