Research map / Label-efficient and robust learning / Recognition and long-tail
Rethinking Evaluation of Infrared Small Target Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hybrid-level metric (hIoU), an error decomposition for IRSTD, an OPDC target-matching strategy, and cross-dataset evaluation, benchmarking 14 deep IRSTD methods on IRSTD1k, SIRST and NUDT.
Problem
Current infrared small target detection (IRSTD) evaluation relies on fragmented pixel- and target-level metrics, aggregate scores that hide failure modes, and single-dataset training/testing that obscures generalization.
Approach
Combine target-level localization IoU and pixel-level segmentation IoU multiplicatively into hIoU, decompose the resulting error into complementary localization and segmentation subtypes, add an overlap-priority matching (OPDC) before centroid-distance compensation, and evaluate cross-dataset.
Key results
- hIoU ranks DNANet first (0.557) on IRSTD1k_TE over MSHNet (0.549), reversing the ranking implied by conventional metrics where MSHNet leads.
- Cross-dataset: IRSTD1k-trained MRF3Net reaches 0.694 hIoU on SIRST_TE, exceeding SIRST-trained UIUNet (0.679) and RPCANet (0.675).
- Error analysis shows interference errors dominate localization error and perception errors dominate segmentation error across models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hierarchical IoU (hIoU) to evaluate joint localization and segmentation rather than relying on isolated conventional metrics | IRSTD detection evaluation · IRSTD1k_TE (models trained on IRSTD1k_TR) | hIoU: 0.557 vs. 0.549 | Mixed |
| Replace distance-only target matching with OPDC matching | IRSTD methods · Cross-dataset test settings in Table 1 | Pd | Helped |
| Replace distance-only target matching with OPDC matching | MSHNet · IRSTD1k_TE (trained on IRSTD1k_TR) | Pd: 0.933 -> 0.936 | Helped |
| Replace distance-only target matching with OPDC matching | MTU-Net · SIRST_TE (trained on IRSTD1k_TR) | Pd: 0.982 -> 1.000 | Helped |
| Replace distance-only target matching with OPDC matching | DNANet · NUDT_TE (trained on IRSTD1k_TR) | Pd: 0.848 -> 0.886 | Helped |
| Replace distance-only target matching with OPDC matching | MSHNet · IRSTD1k_TE (trained on IRSTD1k_TR) | Fa×10^6: 11.539 -> 7.686 | Helped |
| Evaluate models across datasets rather than only on their training dataset | IRSTD methods · Cross-dataset evaluation | hIoU | Hurt |
Surprising findings
- Models trained on IRSTD1k score higher on SIRST test than on IRSTD1k's own test set, suggesting IRSTD1k is the harder benchmark.
- Conventional metrics and hIoU disagree on the best model: MSHNet leads on IoU/F1/Fa while DNANet leads on hIoU.
Limitations
- Synthetic copy-paste test set is used for copyright reasons and only augments IRSTD1k; the test set is not yet released publicly in the main text.
- Matching threshold of 3 pixels and 0.5 IoU are fixed heuristics inherited from prior work.
- Binarization threshold of 0.5 is used by default and its strategy is out of scope.
Experimental setup
Datasets: IRSTD1k SIRST NUDT
Benchmarks: IRSTD1k SIRST NUDT IRSTD1k^AUG_TE
Metrics: IoU_pix nIoU_pix F1_pix Pd Fa hIoU OPDC-Pd OPDC-Fa localization error subtypes segmentation error subtypes
Baselines: ACM FC3Net DNANet ISNet AGPCNet UIUNet RDIAN MTU-Net ABC SeRankDet MSHNet MRF3Net
Builds on: IoU nIoU Pd/Fa metrics scipy linear_sum_assignment
Closest papers by meaning
- Infrared Small Target Detection with Scale and Location Sensitivity (CVPR 2024 · similarity 0.90)
- Rethinking Infrared Small Target Detection: A Foundation Driven Efficient Paradigm (NeurIPS 2026 · similarity 0.89)
- Semi-supervised Infrared Small Target Detection with Thermodynamic-Inspired Uneven Perturbation and Confidence Adaptation (AAAI 2025 · similarity 0.88)
- IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning (AAAI 2024 · similarity 0.88)
- Pinwheel-shaped Convolution and Scale-based Dynamic Loss for Infrared Small Target Detection (AAAI 2025 · similarity 0.88)
- IRMamba: Pixel Difference Mamba with Layer Restoration for Infrared Small Target Detection (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.