Research map / Vision-language and multimodal / Human and embodied modeling
Motion Prior Knowledge Learning with Homogeneous Language Descriptions for Moving Infrared Small Target Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoPKL, a vision-language framework that uses homogeneous language descriptions of target motion (quadrant, region, speed, direction) plus graph attention to learn fine motion priors for moving infrared small target detection.
Problem
Moving infrared small target detection relies on motion cues, but existing multi-frame methods capture only coarse frame-difference motion dominated by background changes, yielding weak fine motion representations.
Approach
Generate structured natural-language descriptions of each target's motion (quadrant, region, speed, direction), embed them with GloVe, align them with visual node features via a distance-weighted motion-vision alignment, build a motion relation graph from mutual-information-weighted region selection, and refine features with a GAT before detection.
Key results
- On ITSDT-15K, MoPKL reaches 79.78% mAP50 and 89.92% F1, versus 76.96% mAP50 and 88.07% F1 for the prior SOTA SSTNet.
- On IRDST, MoPKL reaches 74.54% mAP50 and 86.84% F1, above SSTNet's 71.55% and 85.11%.
- Ablation on ITSDT-15K shows the homogeneous descriptions with speed, direction and quadrant raise mAP50 from 77.48% (general descriptions) to 79.78%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add backbone optimization (ablation) | MoPKL · ITSDT-15K | mAP50: 71.95 -> 76.43 | Helped |
| add backbone optimization (ablation) | MoPKL · IRDST | mAP50: 64.44 -> 68.71 | Helped |
| add simple description embedding and integration (ablation) | MoPKL with backbone optimization · ITSDT-15K | mAP50: 76.43 -> 75.42 | Hurt |
| add simple description embedding and integration (ablation) | MoPKL with backbone optimization · IRDST | mAP50: 68.71 -> 66.51 | Hurt |
| use graph description learning, motion-relation-mining edges, and motion-only nodes instead of cosine-similarity edges (ablation) | MoPKL · ITSDT-15K | mAP50: 76.81 -> 77.62 | Helped |
| use graph description learning, motion-relation-mining edges, and motion-only nodes instead of cosine-similarity edges (ablation) | MoPKL · IRDST | mAP50: 69.35 -> 71.02 | Helped |
| replace motion-only nodes with concatenated motion-vision nodes (ablation) | MoPKL with graph description learning and motion-relation-mining edges · ITSDT-15K | mAP50: 77.62 -> 78.87 | Helped |
| replace motion-only nodes with concatenated motion-vision nodes (ablation) | MoPKL with graph description learning and motion-relation-mining edges · IRDST | mAP50: 71.02 -> 72.27 | Helped |
| add all components, including motion-vision alignment (ablation) | MoPKL · ITSDT-15K | mAP50: 78.87 -> 79.78 | Helped |
| add all components, including motion-vision alignment (ablation) | MoPKL · IRDST | mAP50: 72.27 -> 74.54 | Helped |
| use homogeneous rather than general descriptions (ablation) | MoPKL · ITSDT-15K | mAP50: 77.48 -> 79.78 | Helped |
| add speed estimation, direction calculation, and quadrant division to homogeneous descriptions (ablation) | MoPKL · ITSDT-15K | mAP50: 77.48 -> 79.78 | Helped |
Surprising findings
- Pre-trained BERT embeddings of the descriptions were slightly worse than static GloVe embeddings, which the authors attribute to static vectors being better for graph nodes.
- A fixed-size, non-learned descriptive language scheme was reported to outperform general descriptions even though no text model is fine-tuned.
Limitations
- Low inference speed (10.03 FPS) due to parameter count and FLOPs from multi-frame input.
- Current vision-language models are said to suit only clearly defined classes; the method is tailored to ISTD rather than general use.
- Hyperparameters (tau, s, eta, N, T, K) are set by hand and results are reported on only two datasets.
Experimental setup
Datasets: ITSDT-15K IRDST
Benchmarks: ITSDT-15K IRDST
Metrics: mAP50 Precision Recall F1
Baselines: ACM RISTD ISTDUNet ISNet UIUNet SANet AGPCNet RDIAN DNANet DTUM SIRST5K MSHNet
Builds on: GAT CSPDarknet GloVe FCOS-style decoupled detection head (Ge et al. 2021) SSTNet
Closest papers by meaning
- MOCID: Motion Context and Displacement Information Learning for Moving Infrared Small Target Detection (AAAI 2025 · similarity 0.86)
- Rethinking Infrared Small Target Detection: A Foundation Driven Efficient Paradigm (NeurIPS 2026 · similarity 0.86)
- SAIST: Segment Any Infrared Small Target Model Guided by Contrastive Language-Image Pretraining (CVPR 2025 · similarity 0.85)
- Revisiting Few-Shot Object Detection with Vision-Language Models (NeurIPS 2024 · similarity 0.84)
- Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models (NeurIPS 2025 · similarity 0.84)
- Semi-supervised Infrared Small Target Detection with Thermodynamic-Inspired Uneven Perturbation and Confidence Adaptation (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.