Research map / Detection and segmentation / Open-set and few-shot detection
Multispectral Pedestrian Detection with Sparsely Annotated Label
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAMPD, a teacher-student framework for multispectral (visible/thermal/fusion) pedestrian detection under sparse annotations, combining modality-aware pseudo-label weighting, a positive pseudo-label contrastive loss, and retrieval-based pedestrian patch augmentation with dynamic ground-truth refinement.
Problem
Pedestrian detection in multispectral (visible-thermal) data is degraded when annotations are sparse (some pedestrians missing), and existing sparsely annotated object detection methods do not assess pseudo-label quality or allow the ground-truth pool to grow beyond fixed annotations.
Approach
Weights each modality's detection loss by cosine similarity between pseudo-label and ground-truth features, enhances teacher pseudo-labels via a contrastive loss separating positive/negative pseudo-labels, and augments images with brightness-matched GT pedestrian patches placed in low-saliency regions, while promoting high-quality pseudo-labels into ground truth dynamically.
Key results
- On KAIST at 30% removal, SAMPD reaches 8.56 MR (All) vs 10.47 for Calibrated Teacher and 13.89 for supervised baseline.
- On LLVIP at 70% removal, SAMPD achieves 11.71 MR vs 15.57 for Calibrated Teacher.
- In the fully annotated KAIST setting, SAMPD improves MR from 7.58 to 6.50.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SAMPD instead of Calibrated Teacher | KAIST multispectral detection · KAIST, 30%, 50%, and 70% removal; All, Day, and Night | MR: 10.47 -> 8.56 (30% All); 17.67 -> 15.27 (50% All); 25.48 -> 23.52 (70% All) | Helped |
| Apply SAMPD instead of Calibrated Teacher | LLVIP detection · LLVIP, 30%, 50%, and 70% removal | MR: 9.41 -> 7.65; 12.10 -> 9.03; 15.57 -> 11.71 | Helped |
| Apply SAMPD instead of Calibrated Teacher | LLVIP detection · LLVIP, 30%, 50%, and 70% removal | AP50: 95.27 -> 95.38; 93.75 -> 94.39; 91.51 -> 92.08 | Helped |
| Add MPAW to the supervised detector (ablation) | KAIST 30% removal · KAIST 30%, All | MR: 13.89 -> 10.14 | Helped |
| Add PPE to MPAW (ablation) | KAIST 30% removal · KAIST 30%, All | MR: 10.14 -> 9.65 | Helped |
| Add APRA to MPAW (ablation) | KAIST 30% removal · KAIST 30%, All | MR: 10.14 -> 9.23 | Helped |
| Add PPE and APRA to MPAW (ablation) | KAIST 30% removal · KAIST 30%, All | MR: 9.23 -> 8.56 | Helped |
| Add APRA dynamic pseudo-label refinement (ablation) | APRA with MPAW · KAIST 30%, All | MR: 9.35 -> 9.23 | Helped |
| Add APRA dynamic pseudo-label refinement (ablation) | APRA with MPAW and PPE · KAIST 30%, All | MR: 8.88 -> 8.56 | Helped |
| Use both single- and fusion-modality weights (ablation) | MPAW · KAIST 30%, All | MR: 9.69 -> 8.56 | Helped |
| Set PPE thresholds to 0.9/0.7 rather than 0.9/0.9 (ablation) | SAMPD · KAIST 30%, All | MR: 10.62 -> 8.56 | Helped |
Surprising findings
- Targeted removal of small boxes (their setting) degrades performance more than random removal (13.89 vs 13.04 MR at 30% removal).
Limitations
- APRA provides only image-level guidance and samples patches by overall brightness rather than individual pedestrian characteristics.
- Thresholds (tau1=0.9, tau2=0.7) were chosen empirically via hyperparameter analysis.
Experimental setup
Datasets: KAIST LLVIP
Benchmarks: KAIST LLVIP
Metrics: miss rate (MR) AP50
Baselines: Pseudo label BRL Co-mining SparseDet Calibrated Teacher Robust Teacher MLPD ProbEn MBNet
Builds on: Calibrated Teacher MLPD SSD
Closest papers by meaning
- As Pseudo-Label Free as Possible: Leveraging Adaptive Feature Generation for Sparsely Annotated Object Detection (AAAI 2025 · similarity 0.85)
- Causal Mode Multiplexer: A Novel Framework for Unbiased Multispectral Pedestrian Detection (CVPR 2024 · similarity 0.84)
- Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty (NeurIPS 2026 · similarity 0.84)
- Pedestrian Attribute Recognition as Label-balanced Multi-label Learning (ICML 2024 · similarity 0.83)
- Point-to-Region Loss for Semi-Supervised Point-Based Crowd Counting (CVPR 2025 · similarity 0.83)
- Pedestrian Attribute Recognition: A New Benchmark Dataset and a Large Language Model Augmented Framework (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.