Infrared-visible (IR-VIS) feature matching plays an essential role in cross-modality visual localization, navigation and perception. Along with the rapid development of deep learning techniques, a number of representative image matching methods have been proposed. However, crossmodal feature matching is still a challenging task due to the significant appearance difference. A significant gap for cross-modal feature matching research lies in the absence of standardized benchmarks and metrics for evaluations. In this paper, we introduce a comprehensive cross-modal feature matching benchmark, CM-Bench, which encompasses 30 feature matching algorithms across diverse cross-modal datasets. Specifically, state-of-the-art traditional and deep learning-based methods are first summarized and categorized into sparse, semidense, and dense methods. These methods are evaluated by different tasks including homography estimation, relative pose estimation, and feature-matching-based geo-localization. In addition, we introduce a classification-network-based adaptive preprocessing front-end that automatically selects suitable enhancement strategies before matching. We also present a novel infrared-satellite cross-modal dataset with manually annotated ground-truth correspondences for practical geo-localization evaluation. The dataset and resource will be available at: https://github.com/SLZ98/CM-Bench.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a challenge in computer vision—trying to make sense of images from cameras that see the world in completely different ways.
Sam: Right. This paper introduces CM-Bench, the first thorough test suite for matching features between infrared and visible images. The big puzzle is that while lots of matching methods work well for regular color photos, they fall short when one image shows heat patterns from an infrared camera and the other shows colors from a visible-light camera.
Alex: So this benchmark is basically putting 30 different matching algorithms through the same set of tests to see what actually works across those heat-versus-color gaps?
Sam: Exactly. Infrared cameras pick up heat signatures—like warm engines or people glowing against a cool background—while visible cameras capture colors and textures we see with our eyes. Matching features means finding the same real-world spots, like corners of a building, in both types of pictures even though they look totally different. Without a standard way to compare methods, researchers couldn't tell which ones handle these cross-type image pairs reliably.
Alex: Okay, that sets up the problem. But why does this matter practically—like, where does pairing heat images with color ones come up in real life?
Sam: It's key for things like security cameras at night, drone navigation in bad weather, or self-driving cars fusing sensor data. For instance, a drone's thermal camera might spot a hot vehicle, but to pinpoint its location on a map, it needs to match that heat blob to details in a satellite color photo. Existing color-only matchers fail here because they rely on brightness or patterns that don't line up across heat and light.
Alex: So the heat images make traditional spot-the-keypoint approaches flop. And this CM-Bench levels the playing field by using the same datasets and measures for everyone?
Sam: Yes. They group the 30 algorithms into sparse ones that pick a few key spots—like finding the main landmarks in a photo—semi-dense that cover bigger patches, and dense that map nearly pixel-by-pixel. The paper tests them on tasks like figuring out how images line up flat-on or estimating 3D positions, using 10 datasets from drones, streets, and more. They even add a new dataset, ThermoSat, with infrared-satellite pairs from real flights over farms and cities.
Alex: Interesting. Before this, comparisons were scattered, and no one knew if deep learning fixes or old-school tweaks were best for these modality gaps. So how do they break down the tests into specific tasks to make those comparisons fair?
Sam: They set up four main tasks that ramp up in difficulty. First is homography estimation—aligning flat views like overlapping photos. Next comes relative pose estimation: this figures out the 3D position and angle between two cameras, like knowing exactly where one drone is relative to another from their images.
Alex: Relative pose... so not just flat alignment, but full 3D setup between thermal and visible views.
Sam: Yes. Then geo-localization: dropping a thermal drone image onto a satellite color map to find its exact spot on the ground. ThermoSat makes this possible—pairs from real drone flights over seven areas like farms and cities. They also made a harder version that adds real-world drone wobbles like tilting or height changes.
Alex: Across these, how do the categories stack up—do dense methods pull ahead?
Sam: Dense methods like MINIMA with RoMa topped charts. The logic is that dense matching scans nearly every pixel for similarities based on shapes and semantics, not just heat or color—like checking the outline of a road or building that stays the same no matter the camera type. This bridges modality gaps better than sparse spot-finders, which grab standout points that often vanish in thermal views.
Alex: So denser pixel-wise checks win out because they capture more scene structure that doesn't change between heat and color shifts. On tougher tests like 3D positions or drone localization, does density pull away even more?
Sam: Yes. On street scenes with known camera setups, dense methods get about twice the accuracy of the next best at tight angle limits. For geo-localization on ThermoSat, dense MINIMA(RoMa) achieves over 90% success within 5 meters in steady flights, and holds up under simulated drone tilts. Sparse methods manage decent results but with higher errors, as they lack the broader scene context.
Alex: Huh—that's consistent superiority. But what about real-world costs—like speed or tweaks needed to make them practical?
Sam: Sparse methods run fastest, around 9 to 500 milliseconds per image pair, making them suitable for quick tasks. Dense leaders like MINIMA with RoMa take about 600 milliseconds but deliver the accuracy edge for demanding scenarios. The paper also includes an adaptive preprocessing step—a simple network that picks the best tweak for each pair, like sharpening edges or boosting contrast. This boosts performance further, with gains up to around 12% on some validation sets, without changing the core matching algorithms.
Alex: Okay, so that front-end helps squeeze out extra accuracy, and the trade-off favors density where precision matters more than split-second speed.
Sam: Precisely. The benchmark draws from existing datasets plus the new ThermoSat, so scaling to more varied scenes could refine standings further. Still, it provides a solid, standardized baseline—the first of its kind—for comparing matchers across those tasks, spotlighting dense methods with semantic-rich features as a meaningful path forward for fusing infrared and visible data in security, drones, or vehicles. Overall, CM-Bench establishes a reliable foundation for the field.
Alex: That's a clear step ahead. Thanks for breaking it down, Sam—this makes the logic behind robust IR-VIS matching much sharper. Thanks for listening to ResearchPod.