Research map / Image generation / Image restoration
U-Know-DiffPAN: An Uncertainty-aware Knowledge Distillation Diffusion Framework with Details Enhancement for PAN-Sharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes U-Know-DiffPAN, a diffusion-based PAN-sharpening framework where a frequency-selective attention teacher (FSA-T) with uncertainty map distills into a lightweight student (FSA-S) via uncertainty-weighted losses.
Problem
PAN-sharpening struggles to restore fine high-frequency details; existing diffusion-based PAN-sharpening methods use simple conditioning that underuses PAN and LRMS inputs and are computationally expensive.
Approach
Combines a high-capacity diffusion teacher with frequency-selective attention (FFA encoder conditioning on compact vectors; HQFE decoder with FFT channel attention and SWT cross attention) with uncertainty-aware knowledge distillation, where a predicted pixel-wise uncertainty map weights hard, soft and feature losses so the student focuses on hard regions.
Key results
- On GF2 reduced-resolution, FSA-T reaches PSNR 44.757 and SAM 0.603 vs. CANConv's 43.166 and 0.722.
- On WV3 reduced-resolution, FSA-S achieves PSNR 37.930 and SAM 2.797, surpassing CANConv (37.441, 2.927) and TMDiff (37.477, 2.885).
- FSA-S has 9.115M params and 0.346 TFLOPs versus TMDiff's 153.939M params and 5.517 TFLOPs, with 12.3 s vs 67.5 s inference time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FSA-S instead of CANConv | PAN-sharpening · GF2 reduced-resolution | PSNR: 43.166 -> 44.585 | Helped |
| use FSA-S instead of CANConv | PAN-sharpening · GF2 reduced-resolution | SAM: 0.722 -> 0.624 | Helped |
| use FSA-S instead of TMDiff | PAN-sharpening · WV3 reduced-resolution | PSNR: 37.477 -> 37.930 | Helped |
| use FSA-S instead of TMDiff | PAN-sharpening · WV3 reduced-resolution | SAM: 2.885 -> 2.797 | Helped |
| use FSA-S instead of DCPNet | PAN-sharpening · QB reduced-resolution | PSNR: 38.079 -> 38.361 | Helped |
| use FSA-S instead of DCPNet | PAN-sharpening · QB reduced-resolution | SAM: 4.420 -> 4.337 | Helped |
| add FFA to the encoder (ablation) | FSA-T · GF2 reduced-resolution | ERGAS: 0.661 ± 0.076 -> 0.636 ± 0.077 | Helped |
| add HQFE to the decoder (ablation) | FSA-T · GF2 reduced-resolution | ERGAS: 0.661 ± 0.076 -> 0.556 ± 0.104 | Helped |
| use FFA and HQFE together (ablation) | FSA-T · GF2 reduced-resolution | ERGAS: 0.661 ± 0.076 -> 0.537 ± 0.077 | Helped |
| replace L1 with U-Know loss (ablation) | FSA-S · GF2 full-resolution | HQNR: 0.935 ± 0.020 -> 0.944 ± 0.012 | Helped |
| include uncertainty in KD loss instead of conventional LKD (ablation) | FSA-S · GF2 full-resolution | HQNR: 0.938 ± 0.021 -> 0.944 ± 0.012 | Helped |
Surprising findings
- On WV3 and QB, the distilled student FSA-S slightly outperforms its teacher FSA-T on several metrics.
Limitations
- Diffusion sampling makes inference slower than non-diffusion PAN-sharpening models.
- Teacher FSA-T is computationally heavier than the student (25.5M params, 1.4 TFLOPs).
Experimental setup
Datasets: WorldView-3 (PanCollection) QuickBird (PanCollection) GaoFen-2 (PanCollection)
Benchmarks: WV3 reduced-resolution QB reduced-resolution GF2 reduced-resolution WV3 full-resolution QB full-resolution GF2 full-resolution
Metrics: PSNR SSIM SAM ERGAS SCC Q4 Q8 Dλ Ds HQNR
Baselines: PanNet MSDCNN FusionNet LAGConv S2DBPN DCPNet CANConv PanDiff TMDiff
Builds on: DDPM DDIM knowledge distillation PanDiff TMDiff
Closest papers by meaning
- SSUN-Net: Spatial-Spectral Prior-Aware Unfolding Network for Pan-Sharpening (AAAI 2025 · similarity 0.88)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.88)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.86)
- Privileged Prior Information Distillation for Image Matting (AAAI 2024 · similarity 0.85)
- MDFL: Multi-Domain Diffusion-Driven Feature Learning (AAAI 2024 · similarity 0.85)
- Frequency-Adaptive Pan-Sharpening with Mixture of Experts (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.