Yoonhyung Park, Minji Kim, Sungwon Moon, Jiyoung Lee
3 min
Abstract
Touch supplies the physical grounding needed to perceive intrinsic material properties, such as friction and compliance, that vision alone often cannot resolve. Recent efforts for equipping multimodal LLMs with this tactile sense, however, expose a zero-sum trade-off: the limited parameter budget of compact models forces a choice between acquiring the new sensory modality and preserving the established vision-language reasoning. We present Splash, a mask-isolated tactile alignment learning framework for MLLMs. Splash quantifies the significance of each pretrained parameter, and partitions the parameter space into a dormant and critical subspace. While the frozen critical subspace acts as a stable anchor to safeguard general visual knowledge, Splash updates the isolated dormant subspace to internalize tactile alignment towards LLMs. This selective, non-destructive expansion effectively prevents catastrophic forgetting and ensures non-destructive modality expansion. Extensive experiments show that Splash effectively achieves tactile reasoning without additional inference overhead in the LLM part, demonstrating state-of-the-art performance on visuo-tactile benchmarks, including SSVTP, TVL, and TacQuad, while preserving its original general-purpose capabilities.
Alex: That's a satisfying logic. But I imagine it's not a complete solution yet?
Sam: That's a fair point. The dormant subspace is fixed once training begins. It doesn't adjust in real-time if the robot encounters something unexpected. So if it's never been trained on, say, a wet surface, it can't spontaneously reroute itself to handle that.
Alex: So it's not fully adaptive yet.
Sam: Not yet. The researchers point to what they call "dynamic masking" as a direction for future work — where the robot could re-evaluate which parts of its brain are dormant based on whatever object it's actually touching in the moment. That's a harder problem, but Splash lays the groundwork for it.
Alex: So for now, they've shown you can add a new sense to a compact robot without the usual trade-offs. A solid foundation, with room still to build on.
Sam: That's a good summary. The significance here isn't just the tactile capability itself — it's the principle that a well-targeted, minimal update can outperform a full retraining. That has implications well beyond touch sensing. Thanks for listening to ResearchPod.