TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

How can a robot autonomously locate and identify objects in confined, vision-denied environments using only tactile feedback? The authors address the challenge of creating a unified policy that transitions from broad workspace exploration to precise, contact-driven surface reconstruction without relying on visual sensors.

The authors introduce TACTFUL, a framework that combines behavior cloning (BC) and reinforcement learning (RL) to train a multi-fingered robotic hand. The system is trained entirely on real hardware, avoiding the common pitfalls of sim-to-real gaps. The policy is initialized via BC using teleoperated demonstrations to learn basic contact strategies. It is then refined using Proximal Policy Optimization (PPO) with a dynamic reward schedule. This schedule balances three objectives: global workspace exploration (to find objects), contact-based surface exploitation (to gather data), and shape reconstruction fidelity (to identify the object). The robot uses a transformer-based architecture to process sequences of tactile and proprioceptive data, allowing it to maintain a memory of explored regions and contact points.

TACTFUL achieves a 77% success rate in object identification within confined workspaces, with an average reconstruction error of 0.015 meters. The dynamic reward schedule effectively forces the robot to shift its behavior from initial searching to detailed tactile interaction. By leveraging high-resolution tactile sensors and a learned shape-completion model, the system successfully reconstructs 3D geometries that allow for accurate identification, outperforming baseline methods that rely on hand-crafted heuristics.