Research map / Detection and segmentation / CNN features
SMR-Net: Semantic-Guided Mutually Reinforcing Network for Cross-Modal Image Fusion and Salient Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMR-Net, a lightweight network that jointly trains a visible-infrared image fusion subnetwork (PCI) and a bit-plane-slicing salient object detection subnetwork (BPS) that feeds back semantic guidance to improve both tasks.
Problem
Cross-modal (visible-infrared) image fusion and salient object detection are studied separately, although fused images highlight object structure that could help SOD; the paper targets mutual reinforcement between the two tasks.
Approach
A progressive cross-modal interaction fusion network (local convolutional interaction then global spatial/channel attention) is cascaded with an SOD network that treats the fused image as a third modality, extracts semantic information via bit-plane slicing and deformable convolution, and reweights source features; the SOD loss backpropagates into the fusion network.
Key results
- On VT5000, BPS reports S-measure 0.891, F-measure 0.859, E-measure 0.935, MAE 0.030, the best among compared RGB-T SOD methods on that dataset.
- On TNO, the fusion subnetwork achieves PSNR 66.728, MSE 0.020, VIF 0.743, Qabf 0.547, best on PSNR, MSE and Qabf.
- Ablation on VT5000: adding the BS and DC blocks together raises F-measure from 0.708 to 0.859 and lowers MAE from 0.066 to 0.030.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the bit-plane slicing (BS) block (ablation) | BPS · VT5000 | F-measure: 0.708 -> 0.848 | Helped |
| Add the deformable convolution (DC) block (ablation) | BPS · VT5000 | MAE: 0.066 -> 0.032 | Helped |
| Add both BS and DC blocks (ablation) | BPS · VT5000 | F-measure: 0.708 -> 0.859 | Helped |
| Use Bit4-only instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.859 | Helped |
| Use all four bit planes instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.857 | Helped |
| Use Bit1-only instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.851 | Helped |
| Use Bit2-only instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.855 | Helped |
| Use Bit3-only instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.851 | Helped |
| Use Bit1 and Bit2 instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.853 | Helped |
| Use Bit1, Bit2, and Bit3 instead of no bit-plane configuration (ablation) | BPS · VT5000 | F-measure: 0.847 -> 0.855 | Helped |
Surprising findings
- Using only the fourth (least significant) bit-plane performed best in the ablation, outperforming using all four bit-planes.
Limitations
- Evaluation of the fusion subnetwork is limited to standard datasets and PCI is compared only with fusion baselines on three datasets.
- Training is alternating over only 10 epochs with batch size 4, with limited ablation of the joint training strategy.
Experimental setup
Datasets: VT5000
Benchmarks: TNO RoadScene MSRS VT5000 VT1000 VT821
Metrics: PSNR MSE VIF Qabf S-measure F-measure E-measure MAE
Baselines: FusionGAN PMGI GANMcC SDNet CUFD STDFusionNet TarDAL DIVFusion IRFS MTMR SGDL ADF
Builds on: IRFS MSGD ResNet Deformable Convolution DenseNet
Closest papers by meaning
- Taming Cascaded Mixture-of-Experts for Modality-missing Multi-modal Salient Object Detection (AAAI 2026 · similarity 0.88)
- Self-Supervised Pretraining for RGB-D Salient Object Detection (AAAI 2022 · similarity 0.88)
- Semantic-Aligned Fusion Transformer for One-Shot Object Detection (CVPR 2022 · similarity 0.88)
- Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection (CVPR 2018 · similarity 0.88)
- DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection (AAAI 2025 · similarity 0.87)
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.