HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

3D vision-language models (3D VLMs) often process multi-view inputs that result in thousands of visual tokens. This leads to high computational costs due to redundant observations and uninformative scene regions. The authors investigate how to compress these token sequences without losing the fine-grained spatial details necessary for accurate 3D reasoning.

The authors propose HiSC, a hierarchical spatial clustering framework that operates in two main stages. First, they introduce a Spatial Graph-based Merging (SGraM) strategy, which constructs a connectivity graph based on both geometric proximity and semantic similarity. This allows the model to merge redundant tokens representing the same physical surface or overlapping views before they reach the LLM. Second, they implement a Spatial Clustering-based Pruning (SCluP) paradigm during LLM inference. Instead of pruning tokens independently, SCluP groups tokens into object-centric clusters and performs hierarchical compression, allocating more tokens to important regions while maintaining global scene coverage.

HiSC effectively reduces token redundancy by treating 3D scenes as structured entities rather than flat sequences. Experimental results demonstrate that the framework achieves over 90% token reduction with minimal performance degradation across various 3D reasoning benchmarks. By moving from token-level selection to cluster-level processing, the model maintains better object-level completeness and preserves critical spatial details that traditional, independent token pruning methods often discard.

As 3D VLMs become increasingly important for robotics and spatial understanding, the computational overhead of processing multi-view data remains a significant bottleneck. HiSC provides a training-free, plug-and-play solution that significantly lowers inference costs, making high-performance 3D reasoning more feasible for real-time or resource-constrained applications.