AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection

How can researchers overcome the bottleneck of manual teleoperation in collecting large-scale, physically validated dexterous grasping data? The authors aim to create an end-to-end automated system that generates, executes, and labels grasp attempts on real hardware without human supervision.

AutoDex functions as a closed-loop system consisting of four primary components:

1. Dense Perception: A 20-camera setup provides redundant views to track object poses despite severe hand-object occlusion.
2. Collision-Monitored Execution: A learned residual-torque monitor detects unexpected contacts, allowing the robot to safely abort and recover from failed attempts.
3. Automated Labeling: The system automatically classifies trials as successes or failures based on whether the object is lifted and held for a specified duration.
4. Active Reset: An automated module reorients objects between stable poses to maximize the number of candidates tested per object.

AutoDex successfully collected 3,593 grasp trials across 100 diverse objects. In a direct comparison, the system achieved a 4.8x throughput improvement over human teleoperation (10.3 hours vs. 49.4 hours for 500 trials). Furthermore, grasps retrieved from the AutoDex-validated database demonstrated a 76% real-world success rate, significantly outperforming the 34% success rate of grasps selected based on simulation-only feasibility screening. This confirms that real-world physical validation is essential for identifying grasps that are robust to factors like friction, compliance, and micro-slip.

This work provides a scalable path for building high-quality, real-world datasets for dexterous manipulation. By automating the physical validation loop, researchers can move beyond the limitations of simulation-based grasp synthesis, which often fails to account for complex contact dynamics, and the slow, biased nature of human-collected data.