Research map / Vision-language and multimodal / CLIP and zero-shot
SAIST: Segment Any Infrared Small Target Model Guided by Contrastive Language-Image Pretraining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAIST, which combines a CLIP-based scene-recognition text/visual prompt module (SR-CLIP) with a LoRA-tuned SAM mask decoder guided by an infrared imaging equation (CG-SAM), and releases the image-text MIRSTD dataset for infrared small target detection.
Problem
Infrared small target detection struggles with low SNR targets and complex backgrounds, and image-only methods fail to capture scene context; applying CLIP-SAM to IRSTD faces domain gaps between infrared and natural images.
Approach
Generates text and visual descriptions (object-object similarity, scene-object correlation, image captioning), aligns them with an MMD loss via a cross-modal interaction mechanism to produce prompts, and feeds them into a SAM mask decoder with a Beer-Lambert-based infrared imaging equation to separate target from background.
Key results
- SAIST achieves IoU 95.23, Pd 99.28, Fa 1.31e-6 on NUDT-SIRST, outperforming SCTransNet (IoU 94.09) and other baselines.
- On NUAA-SIRST, SAIST reaches IoU 80.82 and Pd 99.56 with Fa 0.87e-6; removing both SR-CLIP and CG-SAM drops IoU to 76.82.
- On IRSTD-1k, SAIST reports IoU 72.14 and Pd 96.18, the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SR-CLIP (ablation) | SAIST (CLIP + SAM baseline) · NUAA-SIRST | IoU: 76.82 -> 80.02 | Helped |
| add CG-SAM (ablation) | SAIST (CLIP + SAM baseline) · NUAA-SIRST | IoU: 76.82 -> 78.87 | Helped |
| use SR-CLIP instead of CLIP (ablation) | CG-SAM · NUAA-SIRST | IoU: 80.02 -> 80.82 | Helped |
| use SR-CLIP instead of CoOp (ablation) | CG-SAM · NUAA-SIRST | IoU: 80.32 -> 80.82 | Helped |
| use CG-SAM instead of SAM mask decoder (ablation) | SR-CLIP · NUAA-SIRST | IoU: 78.87 -> 80.82 | Helped |
| use CG-SAM instead of HQ-SAM mask decoder (ablation) | SR-CLIP · NUAA-SIRST | IoU: 79.45 -> 80.82 | Helped |
Surprising findings
- SAIST has slightly higher inference time (0.081 s vs 0.079 s) and more trainable parameters (389.57M vs 383.28M) than the SAM+CLIP baseline.
Limitations
- Text descriptions are generated by GPT-4V/ChatGPT and only manually checked, so description quality depends on a closed model.
- Infrared imaging equation relies on a depth estimate F(I) from a ResNet, with simplified assumptions about the scene.
- Reported gains are on three small datasets; no statement about generalization beyond them.
Experimental setup
Datasets: MIRSTD NUAA-SIRST NUDT-SIRST IRSTD-1k
Benchmarks: NUDT-SIRST IRSTD-1k NUAA-SIRST MIRSTD
Metrics: IoU Pd Fa
Baselines: ISNet GCI-Net UIU-Net DNANet Dim2Clear ALCNet ACM IAANet SCTransNet SAM SAM-HQ Top-Hat
Builds on: CLIP SAM CoOp ClipSAM Beer-Lambert law
Closest papers by meaning
- Rethinking Evaluation of Infrared Small Target Detection (NeurIPS 2025 · similarity 0.87)
- Rethinking Infrared Small Target Detection: A Foundation Driven Efficient Paradigm (NeurIPS 2026 · similarity 0.87)
- IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning (AAAI 2024 · similarity 0.86)
- Semi-supervised Infrared Small Target Detection with Thermodynamic-Inspired Uneven Perturbation and Confidence Adaptation (AAAI 2025 · similarity 0.86)
- Motion Prior Knowledge Learning with Homogeneous Language Descriptions for Moving Infrared Small Target Detection (AAAI 2025 · similarity 0.85)
- Infrared Small Target Detection with Scale and Location Sensitivity (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.