RGB imagery offers a practical, low-cost option for Unmanned Aerial/Ground Vehicle (UAV/UGV) survey support in surface-landmine detection, but object detectors remain underexplored in this safety-critical domain. Limited cross-architecture benchmarking and insufficient out-of-distribution (OOD) analysis obscure whether detectors generalize across deployment conditions. This challenge is amplified by the scarcity of public RGB landmine datasets, making SULAND a key benchmark for PFM-1 and PMA-2 detection. However, inspection reveals missing/false annotations, localization errors, inconsistent visibility criteria, visual artifacts, temporal labeling inconsistencies, and an inverted OOD class-ID convention in SULAND. We present SULAND_v2, a refined RGB surface-landmine dataset and benchmark. Preserving original images and splits, we manually revise annotations to ensure completeness, precise localization, label validity, and class consistency. SULAND_v2 contains 33,771 images and 12,433 bounding boxes. We benchmark 35 detector configurations across nine families. Annotation refinement improves YOLOv8 in-distribution (IID) test mAP@50 by 14.6-19.6 percentage points, while fixing the OOD class-ID convention increases mean YOLOv8 OOD mAP@50 by ~25 percentage points. On SULAND_v2, YOLOv12-Small achieves the highest IID mAP@50 (0.908), while RF-DETR-Large yields the strongest OOD performance (0.799 mAP@50, 0.675 recall). Our results demonstrate that high IID accuracy does not guarantee operational readiness. SULAND_v2 provides a reliable benchmark for evaluating domain-shift robustness in RGB-based mine-action survey support.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study on how to improve the reliability of computer vision systems used to detect landmines from drone imagery.
Sam: So the central puzzle here is: why do these AI detectors, which seem to work well in a lab, often fall apart when they're deployed in the real world?
Alex: Precisely. And the answer the researchers keep coming back to isn't the AI model itself—it's the data the model was trained on. Specifically, they found that most datasets in this field contain errors in how objects are labeled or marked. Researchers call this "noisy" data.
Sam: So the "garbage in, garbage out" rule applies here. If the training data is messy, the model just learns the wrong patterns.
Alex: Exactly. And what's notable is that fixing those labels—cleaning up the data—turned out to be more impactful than building a more sophisticated model.
Sam: That's a significant claim. What do these "labels" actually look like in practice?
Alex: When you want to teach an AI to spot a landmine in a photograph, a human expert has to manually draw a box around every mine in thousands of images. That box is the label—it's the AI's instruction manual, saying: this shape, in this location, is what you're looking for. If the box is drawn in the wrong place, or if a mine is missed entirely, the AI starts learning the wrong thing.
Sam: It's like teaching a student with a textbook that has half the answers printed incorrectly. The student isn't the problem—they're just working from bad information. So how did the researchers actually fix this for their dataset?
Alex: They went through the dataset image by image—over 30,000 frames in total. They were looking for three main types of errors: labels that were missing entirely, boxes that didn't properly fit the objects they were marking, and a particularly serious error where the labels for two different mine types had been swapped with each other.
Sam: Swapped labels? So the AI was being told "this is Type A" when it was actually looking at Type B?
Alex: Exactly. And that kind of error is especially damaging because the model doesn't just fail to learn one thing—it actively learns the wrong thing with confidence. Correcting this, along with the other labeling issues, produced performance gains of up to 25 percentage points when the AI was tested in new, unfamiliar environments.
Sam: So the AI wasn't inherently bad at detecting mines. It was being held back by a faulty map of what a mine was supposed to look like.
Alex: That's the right way to think about it. And to check whether this was a fluke, they tested the cleaned data across 35 different model configurations—different AI architectures, different setups. The improvement held consistently across all of them.
Sam: Did some models handle the bad data better than others?
Alex: Not in any meaningful way. What that tells you is that the labeling errors were acting as a ceiling on performance. No matter how well-designed the model was, it couldn't fully compensate for being trained on incorrect information.
Sam: That raises another question. How do we know the model is actually learning what a mine looks like, rather than just memorizing the specific backgrounds or lighting conditions in the training photos?
Alex: That's exactly the right concern. The researchers address it using what's called "out-of-distribution" evaluation. You train the model on data from one environment, then test it on footage from a completely different environment it has never seen before. If the model has genuinely learned to recognize a mine, it should still perform reasonably well. If it was just memorizing the scenery, it falls apart.
Sam: It's like studying for an exam using one textbook, then being tested on questions from a completely different one. It forces you to show whether you actually understand the concept.
Alex: Precisely. And what the researchers found is that even with clean data, a meaningful gap remains between lab performance and real-world results. Fixing the labels is a necessary first step—it gives you an honest baseline—but it doesn't eliminate the underlying challenge of adapting to new terrains, new lighting, new soil types.
Sam: So the paper isn't claiming to have solved the problem. It's saying: before we can make progress, we need to stop measuring ourselves against a broken ruler.
Alex: That's a fair summary. A model trained on sunny, grassy fields in one region will struggle when it's flown over rocky, shadowed terrain somewhere else. And if the training data was inconsistent to begin with, the model has no reliable foundation to generalize from. What this research establishes is that data quality isn't a secondary concern—for safety-critical tasks like this, it's foundational. You cannot meaningfully compare models, or claim real-world progress, if the benchmark itself is unreliable.
Sam: And the stakes here are about as high as they get. These aren't academic benchmarks—they're tools that could one day help clear land that's genuinely dangerous for people to walk on.
Alex: Which is precisely why the researchers argue this kind of careful, unglamorous work—auditing tens of thousands of images, correcting small errors, establishing honest evaluation protocols—matters as much as any advance in the models themselves. Thanks for listening to ResearchPod.