MobileSAM2: Lightweight Segment Anything for Spatial Intelligence

As large foundation models like SAM2 become the standard for segmentation tasks, their massive parameter counts make them impractical for deployment on mobile phones, drones, and other resource-constrained hardware. This paper addresses the challenge of compressing SAM2 into a lightweight, mobile-friendly version without sacrificing its robust temporal and multi-granularity segmentation capabilities.

The authors introduce Hypergraphical Knowledge Distillation (HyperKD), a method that uses hypergraphs to explicitly model and transfer the complex knowledge learned by SAM2. Unlike traditional distillation that focuses on simple feature alignment, HyperKD captures high-order relationships through two specific mechanisms:

• Temporal HyperKD: Constructs hypergraphs across video frames to capture temporal consistency and semantic correlations, ensuring the student model maintains robust tracking and segmentation over time.
• Granularity HyperKD: Builds hypergraphs across different levels of visual concepts (e.g., objects, parts, and subparts) to transfer the model's ability to understand scenes at varying levels of detail.

To create the final MobileSAM2 family, the authors combine this distillation framework with a progressive model contraction strategy. This involves searching for optimal architectural configurations—such as layer counts and embedding dimensions—that minimize the HyperKD loss while significantly reducing the parameter count of the original Hiera-based image encoder.

MobileSAM2 bridges the gap between high-performance foundation models and edge computing. By enabling real-time, accurate segmentation on mobile devices, this work facilitates the deployment of advanced spatial intelligence in robotics, autonomous navigation, and mobile video editing, where low latency and power efficiency are critical requirements.