Research map / Image generation / Image restoration
Hyperspectral Pansharpening via Diffusion Models with Iteratively Zero-Shot Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a guided diffusion scheme for hyperspectral pansharpening that combines a per-image zero-shot guidance network with an iteratively updated neural spatial-spectral decomposition (NSSD) to map a generated RGB detail image to the HR-HSI, using a frozen pre-trained DM.
Problem
Hyperspectral pansharpening from a single PAN and LR-HSI pair; pre-trained diffusion models are trained on RGB images and do not adapt well to hyperspectral data, and fixed spectral mappings from the RGB detail image limit results.
Approach
Guide a frozen pre-trained DM with a zero-shot auxiliary network trained only on the test PAN/LR-HSI pair (via reduced-resolution simulation), and replace the fixed transform matrix with a learnable NSSD (two networks: spatial U-Net and spectral FC) that is alternately updated with the diffusion guidance.
Key results
- On Pavia, the proposed method reaches PSNR 34.227 dB and SSIM 0.926, versus 33.060 dB for PLRDiff (the best baseline).
- On WDC, PSNR 40.503 dB vs 39.703 dB for CNMF; on Chikusei, PSNR 38.450 dB, a 1.274 dB gain over PLRDiff.
- On the real FR1 dataset, QNR 0.832 is the highest, above ZSL at 0.802.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed method instead of PLRDiff | hyperspectral pansharpening · Pavia | PSNR: 33.060 -> 34.227 | Helped |
| use the proposed method instead of CNMF | hyperspectral pansharpening · WDC | PSNR: 39.703 -> 40.503 | Helped |
| use the proposed method instead of PLRDiff | hyperspectral pansharpening · Chikusei | PSNR: 37.176 -> 38.450 | Helped |
| use the proposed method instead of ZSL | real-world hyperspectral pansharpening · FR1 | QNR: 0.802 -> 0.832 | Helped |
| add zero-shot guidance to the fixed-transformation model (ablation) | Pavia pansharpening · Pavia | PSNR: 33.060 -> 33.267 | Helped |
| add NSSD to the regular-fidelity model (ablation) | Pavia pansharpening · Pavia | PSNR: 33.060 -> 33.972 | Helped |
| combine zero-shot guidance and NSSD instead of using either component alone (ablation) | Pavia pansharpening · Pavia | PSNR: 33.267 -> 34.179; 33.972 -> 34.179 | Helped |
| replace proposed zero-shot guidance with fidelity guidance (ablation) | diffusion guidance for Pavia pansharpening · Pavia | PSNR: 32.470 vs 33.374 | Hurt |
| replace proposed zero-shot guidance with total variation guidance (ablation) | diffusion guidance for Pavia pansharpening · Pavia | PSNR: 32.770 vs 33.374 | Hurt |
| use U-Net instead of CONet for the zero-shot guided network (ablation) | Pavia pansharpening · Pavia | PSNR: 34.227 -> 34.015 | Hurt |
| use DSNet instead of CONet for the zero-shot guided network (ablation) | Pavia pansharpening · Pavia | PSNR: 34.227 -> 33.775 | Hurt |
Surprising findings
- On WDC, the proposed method's ERGAS (12.254) is worse than several baselines (e.g., GSA 5.924, HIR-Diff 7.102), despite its best PSNR and SSIM.
Limitations
- NSSD requires more time due to iterative construction of the spectral mapping
- Relies on a fixed pre-trained RGB diffusion model not designed for hyperspectral data
- Evaluated only on simulated degradations for most datasets; real-world evaluation is limited to FR1 with no-reference QNR
Experimental setup
Datasets: Pavia Washington DC Chikusei FR1
Benchmarks: Pavia Washington DC Chikusei FR1
Metrics: PSNR SSIM ERGAS SAM RMSE QNR
Baselines: GSA CNMF HySure ZSL PLRDiff HIR-Diff
Builds on: score-based diffusion models guided diffusion (Dhariwal and Nichol) ZSL (zero-shot pansharpening) PLRDiff HIR-Diff
Closest papers by meaning
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.88)
- EigenSR: Eigenimage-Bridged Pre-Trained RGB Learners for Single Hyperspectral Image Super-Resolution (AAAI 2025 · similarity 0.87)
- FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask (AAAI 2025 · similarity 0.86)
- Enhancing Hyperspectral Images via Diffusion Model and Group-Autoencoder Super-resolution Network (AAAI 2024 · similarity 0.86)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.86)
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.