Research map / Image generation / Image restoration
Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a wavelet-domain diffusion pan-sharpening network that diffuses only the low-frequency wavelet coefficient of the HRMS image (WLDM) and restores high-frequency coefficients from PAN with a HIRM module, achieving better quality and lower inference time than diffusion baselines.
Problem
Pan-sharpening needs to preserve MS spectral information and PAN spatial details, but diffusion-based methods are computationally expensive and slow when generating the full high-resolution multispectral image.
Approach
Decouple high and low frequencies via Haar DWT: a conditional diffusion model (WLDM) generates only the low-frequency wavelet coefficient of the HRMS image starting from the upsampled LRMS image, while a HIRM module restores high-frequency coefficients from PAN with spectral guidance from MS, reducing the diffusion problem's spatial size by 4x.
Key results
- On WorldView-II, the method reaches 42.58 dB PSNR, 0.55 dB above the second-best method (FAME at 42.03 dB).
- On WorldView-III, inference takes 0.118 s and 0.467 GB GPU memory, versus 1.171 s for RDDM and 0.289 s for DDPM.
- Removing HIRM drops GaoFen2 PSNR from 48.78 to 39.74 dB in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed WLDM and HIRM pan-sharpening method instead of FAME | WorldView-II pan-sharpening · WorldView-II | PSNR: 42.0262 -> 42.5780 | Helped |
| use the proposed WLDM and HIRM pan-sharpening method instead of FAME | WorldView-III pan-sharpening · WorldView-III | PSNR: 30.9903 -> 31.0295 | Helped |
| use the proposed WLDM and HIRM pan-sharpening method instead of CANNet | GaoFen2 pan-sharpening · GaoFen2 | PSNR: 47.9369 -> 48.7757 | Helped |
| add HIRM to the method (ablation) | GaoFen2 pan-sharpening · GaoFen2 | PSNR: 39.7350 -> 48.7757 | Helped |
| add diffusion loss Ldiff (ablation) | the proposed method · GaoFen2 | PSNR: 46.5993 -> 48.7757 | Helped |
| add detail loss Ldetail (ablation) | the proposed method · GaoFen2 | PSNR: 47.5214 -> 48.7757 | Helped |
| add content loss Lcontent (ablation) | the proposed method · GaoFen2 | PSNR: 44.7655 -> 48.7757 | Helped |
| use wavelet scale K = 2 instead of K = 1 (ablation) | the proposed method · GaoFen2 | PSNR: 48.7757 -> 42.8220 | Hurt |
| use wavelet scale K = 3 instead of K = 1 (ablation) | the proposed method · GaoFen2 | PSNR: 48.7757 -> 40.7969 | Hurt |
Surprising findings
- Increasing the wavelet decomposition scale from K=1 to K=2 or K=3 sharply degrades performance (PSNR 48.78 to 42.82 and 40.80 dB on GaoFen2).
Limitations
- Diffusion sampling still needs multiple steps (10 implicit steps), so it remains slower than non-diffusion deep models such as CANNet and DISPNet
- Training data are simulated via Wald protocol downsampling rather than real paired data
- Performance drops when the wavelet decomposition is applied more than once
Experimental setup
Datasets: WorldView-II WorldView-III GaoFen2
Benchmarks: WorldView-II WorldView-III GaoFen2 full-resolution GaoFen2 real dataset
Metrics: PSNR SSIM SAM ERGAS Dλ DS QNR inference time GPU memory
Baselines: Brovey GS IHS GFPCA CANNet DISPNet FAME DDPM DDIM RDDM DDRF
Builds on: DDPM DDIM Wald protocol U-Net
Closest papers by meaning
- Progressive High-Frequency Reconstruction for Pan-Sharpening with Implicit Neural Representation (AAAI 2024 · similarity 0.90)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.88)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.88)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.87)
- Multi-Domain Multi-Scale Diffusion Model for Low-Light Image Enhancement (AAAI 2024 · similarity 0.87)
- Relay Diffusion: Unifying diffusion process across resolutions for image synthesis (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.