Research map / Detection and segmentation / Open-set and few-shot detection
Alignment-Free RGB-T Salient Object Detection: A Large-Scale Dataset and Progressive Correlation Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds UVT20K, a 20,000-pair unaligned RGB-T salient object detection dataset with train/test splits and multiple annotations, and proposes PCNet, which explicitly aligns regions via a semantics-adapted homography estimator and models inter- and intra-modal correlations.
Problem
Salient object detection from unaligned visible-thermal image pairs is hard because existing RGB-T benchmarks are manually aligned and small, and existing methods handle large spatial/scale misalignment poorly.
Approach
Fine-tune a pretrained multi-modal homography estimator (IHN) with a semantics-guided adapter (S-Adapter) to explicitly align common RGB-thermal regions, then progressively model inter-modal correlation on aligned regions followed by intra-modal correlation over the whole RGB feature.
Key results
- PCNet reaches E_m 0.897, S_m 0.872, F_m 0.822 on UVT20K, the best among 13 compared RGB-T SOD methods.
- On UVT2000, PCNet improves over the sub-optimal CAVER by 6.2% (E_m), 2.8% (S_m) and 7.8% (F_m) as reported in the paper.
- Removing SHE drops UVT20K E_m from 0.897 to 0.875, showing explicit alignment contributes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SHE for homography alignment (ablation) | PCNet · UVT20K test | F_m: 0.786 -> 0.822 | Helped |
| add S-Adapter (ablation) | PCNet · UVT20K test | F_m: 0.798 -> 0.822 | Helped |
| use parameter-efficient S-Adapter instead of full fine-tuning (ablation) | PCNet · UVT20K test | F_m: 0.810 -> 0.822 | Helped |
| add IIMC instead of feature summation (ablation) | PCNet · UVT20K test | F_m: 0.782 -> 0.822 | Helped |
| add intra-modal correlation (ablation) | PCNet · UVT20K test | F_m: 0.791 -> 0.822 | Helped |
| add semantic guidance to SHE and IIMC (ablation) | PCNet · UVT20K test | F_m: 0.804 -> 0.822 | Helped |
| use thermal input in addition to RGB (ablation) | PCNet · UVT20K test | F_m: 0.747 -> 0.822 | Helped |
| train with UVT20K instead of un-VT5000 | PCNet · UVT2000 | E_m: 0.799 -> 0.851 | Helped |
Surprising findings
- Full fine-tuning of the homography estimator (w/ FFT) performs worse than the frozen-estimator adapter variant, which the authors attribute to destroying pretrained capability.
Limitations
- Only 2D homography alignment; the authors note future work on varying viewpoints and depth of field.
- PCNet is slightly weaker than SACNet/CAVER on the aligned VT5000 E_m and F_m metrics.
Experimental setup
Datasets: UVT20K VT5000 un-VT5000 UVT2000 un-VT821 un-VT1000 VT821 VT1000
Benchmarks: UVT20K UVT2000 un-VT5000 un-VT1000 un-VT821 VT5000 VT1000 VT821
Metrics: E-measure S-measure F-measure
Baselines: SACNet LAFB CAVER LSNet MCFNet HRTransNet DCNet TNet OSRNet SwinNet CGFNet CSRNet
Builds on: IHN Swin Transformer VT5000 CAVER
Closest papers by meaning
- Breaking Alignment Barriers: TPS-Driven Semantic Correlation Learning for Alignment-Free RGB-T Salient Object Detection (AAAI 2026 · similarity 0.93)
- Uncertainty-Aware Modality Fusion for Unaligned RGB-T Salient Object Detection (CVPR 2026 · similarity 0.91)
- DVSOD: RGB-D Video Salient Object Detection (NeurIPS 2023 · similarity 0.90)
- RGB-D Salient Object Detection via 3D Convolutional Neural Networks (AAAI 2021 · similarity 0.90)
- DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection (AAAI 2025 · similarity 0.88)
- HSOD-BIT-V2: A Challenging Benchmark for Hyperspectral Salient Object Detection (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.