SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection

Dexterous manipulation requires the ability to detect slip—the loss of friction between a gripper and an object—before a grasp fails. Existing tactile sensors often struggle with two major issues: high detection latency, which prevents timely corrective action, and poor generalization, where models fail when transferred to new objects, different sensor units, or new robotic hands. This paper introduces SlipSense to address these limitations through a multimodal approach.

The authors developed the TacV5 sensor, a compact module that integrates a 32x32 piezoresistive array (capturing spatial pressure distribution) and a 3-axis MEMS accelerometer (capturing friction-induced vibrations). The SlipSense framework processes these two streams using modality-specific encoders and a causal cross-attention mechanism. This design allows the model to fuse spatial and temporal cues at 240 Hz, effectively filtering out noise that would confound a single-modality system. For instance, while pressure arrays can be confused by grasp variations, and accelerometers can be triggered by environmental vibrations, the fusion of both allows the model to distinguish true slip from these artifacts.

SlipSense achieves a high Macro F1 score of 96.7% and maintains a false-positive rate below 1.6%. A critical contribution is the system's ability to perform zero-shot transfer: a model trained on a simple parallel-jaw gripper (UMI) successfully generalized to a more complex, multi-fingered dexterous hand (Tesollo) without any retraining. The system is also highly responsive, detecting 76% of slip events within 23.1 ms, which is sufficient for real-time closed-loop control. The authors demonstrate that the two modalities are truly complementary, as fusion significantly outperforms either modality used in isolation.

This work provides a robust, generalizable solution for tactile slip detection that is ready for real-world robotic integration. By demonstrating that a single model can transfer across different hardware platforms and objects, the authors move the field closer to plug-and-play tactile sensing for dexterous robots. The low-latency performance is particularly important for high-speed manipulation tasks where even a few milliseconds of delay can lead to dropped objects.