SULAND v2 Refines RGB Landmine Detection Data and Exposes Domain-Shift Gaps

Lead

RGB cameras are attractive for UAV and UGV mine-action survey support because they are inexpensive, lightweight, and easy to integrate. But in a safety-critical task such as surface-landmine detection, a detector’s headline accuracy is only meaningful if the benchmark itself is reliable and if the model is tested beyond the conditions it was trained on. SULAND v2 addresses that gap by auditing and rebuilding a key public RGB landmine dataset while preserving its original images and splits.

Core points

• A refinement of an existing benchmark: The authors did not create a new image collection. Instead, they kept the original SULAND imagery and data partitions, then manually revised the annotations in Label Studio so that v1 and v2 remain directly comparable.
• Multiple annotation issues were corrected: The audit found missing and false labels, inaccurate bounding boxes, inconsistent rules for partially visible objects, visual artifacts, and temporal labeling inconsistencies.
• A critical OOD convention was fixed: The original benchmark contained an inverted class-ID convention in the out-of-distribution split, a problem that could substantially distort robustness evaluation.
• Dataset scale: SULAND v2 contains 33,771 RGB images and 12,433 bounding boxes, focusing on surface detection of PFM-1 and PMA-2 landmines.
• Broad detector comparison: The benchmark covers 35 configurations across nine detector families, including one-stage, two-stage, transformer-based, and open-vocabulary approaches under a unified protocol.

What changed in the results

The impact of data quality is not subtle. After annotation refinement, YOLOv8’s in-distribution test mAP@50 rises by 14.6 to 19.6 percentage points. Fixing the OOD class-ID convention alone increases mean YOLOv8 OOD mAP@50 by about 25 percentage points. In other words, benchmark noise was large enough to reshape the perceived capability of a detector.

On the refined benchmark, YOLOv12-Small reaches the strongest IID result with 0.908 mAP@50. Under domain shift, however, RF-DETR-Large leads with 0.799 OOD mAP@50 and 0.675 recall. This split outcome is the main lesson: the best model on familiar data is not automatically the best model under shifted deployment conditions. The authors also report that model size is not a dependable proxy for robustness.

Why it matters

SULAND v2 is valuable not only as a cleaner dataset, but also as a reminder that evaluation design can make or break trust in applied AI. For UAV- and UGV-based mine survey support, models must handle changes in terrain, visibility, sensor viewpoint, and collection conditions. A high IID score may show that a detector learned the benchmark distribution, but it does not prove operational readiness.

For researchers, SULAND v2 offers a more reliable public benchmark for RGB-based surface-landmine detection. For practitioners, it reinforces a stricter deployment standard: domain-shift testing should be treated as a required step, not an optional add-on.

Source: Hugging Face Daily Papers