AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Prior

Robotic manipulation of non-Lambertian objects—such as transparent containers or highly reflective metallic surfaces—is frequently hindered by corrupted or missing raw depth data. Existing depth completion methods often prioritize minimizing global pixel-wise error metrics (like RMSE), which may not prevent catastrophic failures in object geometry that lead to invalid grasp poses. This paper asks how to improve the structural integrity and physical plausibility of depth maps to ensure reliable robotic grasping in challenging sensing conditions.

The authors propose AISPO, a depth completion framework that integrates three complementary information streams: RGB color/texture, raw depth, and a learned affine-invariant shape prior. The framework utilizes a two-stage training paradigm. First, a shape-prior autoencoder is trained to learn a latent representation of object geometry that is invariant to scale and shift. Second, the full model uses three parallel encoders—an RGB encoder (DINOv2), a raw depth encoder (Swin-Transformer), and the frozen shape-prior encoder—to extract multi-scale features. These features are fused via cross-attention transformers and decoded into a high-fidelity, restored depth map.

AISPO demonstrates that enforcing geometric consistency through learned shape priors is more effective for robotic manipulation than traditional pixel-wise optimization. In extensive benchmark evaluations, the model shows strong generalization to unseen objects and novel scenes. Crucially, real-world robotic experiments confirm that the improved structural integrity of the predicted depth maps directly translates to higher grasp success rates for transparent and specular objects, where baseline methods often fail to produce usable geometric data.

Reliable depth perception is a bottleneck for autonomous robots operating in unstructured environments like kitchens or warehouses. By effectively handling the "catastrophic failures" associated with transparent and reflective surfaces, AISPO bridges the gap between perception and successful physical interaction, enabling robots to manipulate a wider variety of common household items without requiring scene-specific training or computationally expensive multi-view optimization.