Research map / Image generation / Image restoration
PanAdapter: Two-Stage Fine-Tuning with Spatial-Spectral Priors Injecting for Pansharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PanAdapter, a two-stage parameter-efficient fine-tuning framework that adapts a pre-trained EDSR CNN and a frozen pre-trained IPT ViT via cascaded dual-branch adapters for pansharpening.
Problem
Pansharpening (fusing low-resolution multispectral and high-resolution panchromatic images) suffers from small datasets (500 to 10,000 images) that limit deep models, and existing PEFT methods adapt poorly from natural-image restoration models to satellite pansharpening.
Approach
Two-stage fine-tuning: first fine-tune a pre-trained EDSR CNN with Local Prior Extraction modules to get spatial and spectral priors at two scales, then inject these priors through cascaded adapters (CTF and CTI modules) into a frozen pre-trained IPT ViT, training only the small adapters and LPE modules.
Key results
- On WV3 reduced-resolution, PanAdapter reaches PSNR 39.473 and Q8 0.923, the best among compared methods (CANNet PSNR 38.908).
- With 2.19M trainable parameters (1.85% of the 115.2M full fine-tune), PanAdapter achieves SAM 2.917 and ERGAS 2.149 on WV3, beating full fine-tuning (SAM 3.288, ERGAS 2.473) and other PEFT methods.
- On QB reduced-resolution, PanAdapter attains the best Q4 (0.938) and ERGAS (3.593), while CANNet has slightly better SAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PanAdapter instead of CANNet | pansharpening model · WV3 reduced-resolution | PSNR: 38.908 ± 2.749 -> 39.473 ± 2.626 | Helped |
| Use PanAdapter instead of CANNet | pansharpening model · WV3 full-resolution | Dλ: 0.0165 ± 0.0074 -> 0.0173 ± 0.0076 | Hurt |
| Use PanAdapter instead of CANNet | pansharpening model · GF2 reduced-resolution | SAM: 0.688±0.129 -> 0.702±0.143 | Hurt |
| Use PanAdapter instead of CANNet | pansharpening model · QB reduced-resolution | ERGAS: 3.652±0.327 -> 3.593±0.356 | Helped |
| Remove two-stage training (ablation) | PanAdapter · WV3 | SAM: 2.917$\pm$0.560 -> 3.573$\pm$0.682 | Hurt |
| Remove two-stage training (ablation) | PanAdapter · WV3 | ERGAS: 2.149$\pm$0.492 -> 2.576$\pm$0.619 | Hurt |
| Remove two-stage training (ablation) | PanAdapter · WV3 | Q8: 0.906$\pm$0.095 -> 0.923$\pm$0.081 | Hurt |
| Replace INR with common convolutional upsampling (ablation) | PanAdapter · WV3 | SAM: 2.917$\pm$0.560 -> 2.957$\pm$0.640 | Hurt |
| Replace INR with common convolutional upsampling (ablation) | PanAdapter · WV3 | ERGAS: 2.149$\pm$0.492 -> 2.381$\pm$0.916 | Hurt |
| Replace INR with common convolutional upsampling (ablation) | PanAdapter · WV3 | Q8: 0.916$\pm$0.090 -> 0.923$\pm$0.081 | Hurt |
| Remove CTF module (ablation) | PanAdapter · WV3 | SAM: 2.917$\pm$0.560 -> 2.942$\pm$0.611 | Hurt |
| Remove CTI module (ablation) | PanAdapter · WV3 | SAM: 2.917$\pm$0.560 -> 3.301$\pm$0.642 | Hurt |
Surprising findings
- On GF2 reduced-resolution, PanAdapter is not best on SAM or ERGAS (CANNet/HMPNet are better on ERGAS), and on the QB full-resolution comparison it is only marginally ahead.
Limitations
- Results on GF2 and QB are not uniformly best across metrics.
- Relies on simulated reduced-resolution data via Wald's protocol since ground truth is unavailable at full resolution.
Experimental setup
Datasets: WorldView-3 (simulated via Wald's protocol) QuickBird (simulated) GaoFen-2 (simulated) DIV2K
Benchmarks: WV3 QB GF2
Metrics: PSNR Q8 SAM ERGAS D_lambda D_S QNR
Baselines: PanNet FusionNet LAGConv PMACNet BiMPan HMPNet PanDiff PanMamba CANNet LoRA Adapter LST
Builds on: ViT-Adapter AdaptFormer LST IPT EDSR
Closest papers by meaning
- Dual-Granularity Semantic Guided Sparse Routing Diffusion Model for General Pansharpening (CVPR 2025 · similarity 0.86)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.86)
- SSUN-Net: Spatial-Spectral Prior-Aware Unfolding Network for Pan-Sharpening (AAAI 2025 · similarity 0.85)
- A General Adaptive Dual-level Weighting Mechanism for Remote Sensing Pansharpening (CVPR 2025 · similarity 0.85)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.85)
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.