MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models

How can hyperspectral foundation models be effectively adapted to downstream classification tasks when the input data from different sensors vary significantly in spectral band configurations, without discarding critical spectral information or incurring high computational costs?

The authors propose the Multi-Branch efficient fine-tuning framework (MBTI). Instead of compressing or reshaping hyperspectral images (HSIs) to fit a fixed model input, MBTI divides the full-band data into continuous spectral subsets. When the final subset is smaller than the required input size, the framework uses a band-reuse mechanism to maintain continuity without resorting to zero-padding. Each branch is processed by a frozen foundation model backbone augmented with independent Low-Rank Adaptation (LoRA) modules. Finally, a multi-branch channel attention fusion module integrates the features from these branches to produce the final classification.

MBTI successfully preserves the integrity of the original spectral data while adapting to diverse sensor configurations. By utilizing branch-specific LoRA modules, the framework achieves competitive classification performance on benchmark datasets while keeping the number of trainable parameters extremely low (approximately 2.33%–2.36%). This demonstrates that the multi-branch approach effectively captures discriminative features across different spectral intervals without the need for full model fine-tuning.

Traditional adaptation methods often sacrifice spectral resolution or continuity to meet the rigid input requirements of pre-trained models, which can degrade performance in tasks where subtle spectral differences are vital. MBTI offers a parameter-efficient, sensor-agnostic solution that maximizes the utility of foundation models in hyperspectral remote sensing, making it easier to deploy these models in resource-constrained or data-scarce environments.