Research map / Detection and segmentation / CNN features

DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection

AAAI 2025 · Detection and segmentation › CNN features · approach: RGB-D salient object detection · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Reformulates multi-modal salient object detection (RGB, RGB-D, RGB-T) as conditional diffusion mask generation by fine-tuning Stable Diffusion with a PVT image encoder, a SOD-ControlNet with Feature Adaptive Network, and a Feature Injection Attention Network.

Problem

Unified multi-modal salient object detection across RGB, RGB-D and RGB-T inputs, where deterministic discriminative models struggle with cross-modal fusion, edge precision, and overconfident predictions.

Approach

Treats SOD as a noise-to-mask conditional diffusion process on a pretrained Stable Diffusion, injecting depth or thermal conditions at all resolutions through SOD-ControlNet with instance-normalized adaptive modulation (FAN), and injecting global image semantics via cross-attention (FIAN).

Key results

• On DUTS-TE, DiMSOD reports F-beta 0.918 and MAE 0.025, versus 0.909 and 0.026 for the second-best M3Net among the RGB comparisons.
• On RGB-D NJUD, DiMSOD reports MAE 0.028 and F-beta 0.947 (table row for OURS), and the text claims MAE and F-beta gains of 6.48% and 1.15% over the second-best method.
• Ablation on DUTS-TE shows SOD-ControlNet lowers MAE from 0.037 (SD+PVT+CNet) to 0.026, and adding FIAN further yields 0.025 with F-beta 0.918.

Measured effects reported in the paper

Surprising findings

• The authors report that DiMSOD's inference time shows no notable advantage over other methods, a limitation of diffusion-based generation.
• Initializing inference with standard Gaussian noise gave better results than multi-resolution noise even though training used multi-resolution noise.

Limitations

• Inference time is not notably better than competing methods, a limitation of diffusion models.
• On RGB-T VT1000, DiMSOD has mixed wins and losses against VSCode.
• Training depends on relatively coarse SOD benchmark data with sparse ground truth requiring self-diffusion tricks.

Experimental setup

Datasets: DUTS-TR VT5000 NJUD NLPR DUTLF-Depth DUT-OMRON ECSSD PASCAL-S HKU-IS DUTS-TE DUTLF-Depth test NJUD test

Benchmarks: DUT-OMRON ECSSD PASCAL-S HKU-IS DUTS-TE DUTLF-Depth NJUD NLPR SIP LFSD VT821 VT1000

Metrics: F-beta MAE E-measure S-measure

Baselines: MMNet LAFB VSCode F3Net MINet U2Net PFSNet VST EDN SHNet SRfor M3Net

Builds on: Stable Diffusion ControlNet DDIM SeeCoder Pyramid Vision Transformer

Closest papers by meaning

• SimpleDiffusion: A Lightweight and Efficient Conditional Diffusion Model for Multi-Modal Salient Object Detection (AAAI 2026 · similarity 0.93)
• DVSOD: RGB-D Video Salient Object Detection (NeurIPS 2023 · similarity 0.91)
• Deep RGB-D Saliency Detection With Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR 2021 · similarity 0.89)
• Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection (ICLR 2022 · similarity 0.89)
• Alignment-Free RGB-T Salient Object Detection: A Large-Scale Dataset and Progressive Correlation Network (AAAI 2025 · similarity 0.88)
• SMR-Net: Semantic-Guided Mutually Reinforcing Network for Cross-Modal Image Fusion and Salient Object Detection (AAAI 2025 · similarity 0.87)