Liangzheng Sun, Mengfan He, Xingyu Shao, Binbin Li, Zhiqiang Yan, Chunyu Li, Ziyang Meng, Fei Xing
6 min
Abstract
Infrared-visible (IR-VIS) feature matching plays an essential role in cross-modality visual localization, navigation and perception. Along with the rapid development of deep learning techniques, a number of representative image matching methods have been proposed. However, crossmodal feature matching is still a challenging task due to the significant appearance difference. A significant gap for cross-modal feature matching research lies in the absence of standardized benchmarks and metrics for evaluations. In this paper, we introduce a comprehensive cross-modal feature matching benchmark, CM-Bench, which encompasses 30 feature matching algorithms across diverse cross-modal datasets. Specifically, state-of-the-art traditional and deep learning-based methods are first summarized and categorized into sparse, semidense, and dense methods. These methods are evaluated by different tasks including homography estimation, relative pose estimation, and feature-matching-based geo-localization. In addition, we introduce a classification-network-based adaptive preprocessing front-end that automatically selects suitable enhancement strategies before matching. We also present a novel infrared-satellite cross-modal dataset with manually annotated ground-truth correspondences for practical geo-localization evaluation. The dataset and resource will be available at: https://github.com/SLZ98/CM-Bench.
Alex: Relative pose... so not just flat alignment, but full 3D setup between thermal and visible views.
Sam: Yes. Then geo-localization: dropping a thermal drone image onto a satellite color map to find its exact spot on the ground. ThermoSat makes this possible—pairs from real drone flights over seven areas like farms and cities. They also made a harder version that adds real-world drone wobbles like tilting or height changes.
Alex: Across these, how do the categories stack up—do dense methods pull ahead?
Sam: Dense methods like MINIMA with RoMa topped charts. The logic is that dense matching scans nearly every pixel for similarities based on shapes and semantics, not just heat or color—like checking the outline of a road or building that stays the same no matter the camera type. This bridges modality gaps better than sparse spot-finders, which grab standout points that often vanish in thermal views.
Alex: So denser pixel-wise checks win out because they capture more scene structure that doesn't change between heat and color shifts. On tougher tests like 3D positions or drone localization, does density pull away even more?
Sam: Yes. On street scenes with known camera setups, dense methods get about twice the accuracy of the next best at tight angle limits. For geo-localization on ThermoSat, dense MINIMA(RoMa) achieves over 90% success within 5 meters in steady flights, and holds up under simulated drone tilts. Sparse methods manage decent results but with higher errors, as they lack the broader scene context.
Alex: Huh—that's consistent superiority. But what about real-world costs—like speed or tweaks needed to make them practical?
Sam: Sparse methods run fastest, around 9 to 500 milliseconds per image pair, making them suitable for quick tasks. Dense leaders like MINIMA with RoMa take about 600 milliseconds but deliver the accuracy edge for demanding scenarios. The paper also includes an adaptive preprocessing step—a simple network that picks the best tweak for each pair, like sharpening edges or boosting contrast. This boosts performance further, with gains up to around 12% on some validation sets, without changing the core matching algorithms.
Alex: Okay, so that front-end helps squeeze out extra accuracy, and the trade-off favors density where precision matters more than split-second speed.
Sam: Precisely. The benchmark draws from existing datasets plus the new ThermoSat, so scaling to more varied scenes could refine standings further. Still, it provides a solid, standardized baseline—the first of its kind—for comparing matchers across those tasks, spotlighting dense methods with semantic-rich features as a meaningful path forward for fusing infrared and visible data in security, drones, or vehicles. Overall, CM-Bench establishes a reliable foundation for the field.
Alex: That's a clear step ahead. Thanks for breaking it down, Sam—this makes the logic behind robust IR-VIS matching much sharper. Thanks for listening to ResearchPod.