Research map / Image generation / Image restoration
SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSDiff, a dual-branch (spatial/spectral) conditional diffusion model for remote sensing pansharpening with an alternating projection fusion module, frequency modulation inter-branch module, and LoRA-like branch-wise alternating fine-tuning.
Problem
Pansharpening fuses high-resolution PAN images with low-resolution multispectral images to produce high-resolution MS images; existing DDPM-based pansharpening methods do not model the distinct spatial and spectral features needed for the task.
Approach
Decomposes pansharpening into spatial and spectral subspace components learned by separate diffusion branches, fused via an alternating projection (attention-like) module, with a frequency modulation module between branches and branch-wise alternating fine-tuning that updates one branch at a time without adding parameters.
Key results
- On WV3 reduced resolution, SSDiff reports SAM 2.8466 and ERGAS 2.1086, best among compared methods (DCFNet second-best at 3.0264 and 2.1588).
- On WV3 full resolution, SSDiff achieves HQNR 0.9565 versus FusionNet's 0.9406 (second-best among DL methods).
- On QuickBird reduced resolution, SSDiff achieves SAM 4.4640 and ERGAS 3.6320; removing FMIM raises WV3 SAM from 2.8646 to 2.9798.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SSDiff instead of DCFNet | Pansharpening · WorldView-3 reduced resolution | SAM: 3.0264±0.7397 -> 2.8466±0.5285 | Helped |
| use SSDiff instead of LAGConv | Pansharpening · GaoFen-2 reduced resolution | SAM: 0.7859±0.1478 -> 0.6694±0.1244 | Helped |
| use SSDiff instead of DCFNet | Pansharpening · QuickBird reduced resolution | SAM: 4.5383±0.7397 -> 4.4640±0.7473 | Helped |
| use SSDiff instead of DCFNet | Models trained on WorldView-3 · WorldView-2 reduced resolution | SAM: 5.6194±0.6039 -> 5.0647±0.5634 | Helped |
| use a single spatial branch with coupled inputs (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.8466±0.5241 -> 3.3612±0.6497 | Hurt |
| use a single spectral branch with coupled inputs (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.8466±0.5241 -> 3.0598±0.5560 | Hurt |
| replace inter-branch interaction with concatenation (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.8466±0.5241 -> 3.4040±0.6088 | Hurt |
| replace APFM with the LoRA-style operation (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.8466±0.5241 -> 3.4871±0.5821 | Hurt |
| add FMIM (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.9798±0.6060 -> 2.8646±0.5241 | Helped |
| alternately fine-tune the spatial and spectral branches (ablation) | SSDiff · WorldView-3 reduced resolution | SAM: 2.8646±0.5241 -> 2.8466±0.5285 | Helped |
Surprising findings
- Sampling for a single WV3 picture takes 89.136 seconds, a substantial inference cost for a diffusion method.
Limitations
- Code promised only after possible acceptance; not released at submission.
- Slow sampling (about 89 s per image on WV3).
- Frequency modulation constants (1.2, 1.4, 1.6) are hand-set and not justified in depth.
Experimental setup
Datasets: WorldView-3 (Pancollection) GaoFen-2 (Pancollection) QuickBird (Pancollection) WorldView-2 (generalization test)
Benchmarks: WorldView-3 GaoFen-2 QuickBird WorldView-2
Metrics: SAM ERGAS Q2n SCC D_lambda D_s HQNR QNR
Baselines: BDSD-PC MTF-GLP-FS BT-H PNN DiCNN MSDCNN FusionNet CTINN LAGConv MMNet DCFNet PanDiff
Builds on: DDPM DDIM LoRA FusionNet self-attention
Closest papers by meaning
- HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectral Images Fusion (ICML 2025 · similarity 0.88)
- Hyperspectral Pansharpening via Diffusion Models with Iteratively Zero-Shot Guidance (CVPR 2025 · similarity 0.88)
- Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening (CVPR 2024 · similarity 0.87)
- S2CycleDiff: Spatial-Spectral-Bilateral Cycle-Diffusion Framework for Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.86)
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.86)
- FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.