Research map / Image generation / Image restoration
Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WFANet, a wavelet-based pansharpening network whose Multi-Frequency Fusion Attention (MFFA) builds a Frequency-Query, Spatial-Key, Fusion-Value attention triplet on DWT sub-bands, combined with an FAB-based spatial detail module and a two-scale wavelet pyramid.
Problem
Pansharpening fuses a high-resolution PAN image with a low-resolution multispectral image, but spatial-domain methods lose detail and existing frequency-domain methods suffer from frequency mixing and information loss during fusion.
Approach
Use DWT to cleanly separate PAN frequency sub-bands, assign physically motivated roles (frequency sub-bands as queries, the LL low-frequency spatial content as keys, and LRMS-PAN fusion features as values) in an attention block, reconstruct with lossless IDWT, and fuse progressively over a wavelet pyramid.
Key results
- On reduced-resolution WV3, the proposed method reaches PSNR 39.345 dB, SAM 2.849, ERGAS 2.093, Q8 0.922, beating the second-best (U2Net 39.117 dB) by 0.228 dB.
- On GF2 reduced-resolution, PSNR improves by 0.417 dB over the second-best method (CANNet 43.496 dB), reaching 43.913 dB.
- On WV3 full-resolution, the method attains the best HQNR of 0.957, with D_lambda 0.017 and D_s 0.027.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use frequency-domain queries instead of a spatial-domain query (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 38.884 -> 39.345 | Helped |
| Use P_LL as Spatial-Key instead of features from P after convolution (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 39.156 -> 39.345 | Helped |
| Include P_LL spatial information in Fusion-Value along with LRMS (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 38.921 -> 39.345 | Helped |
| Use MFFA attention instead of the convolutional-network replacement (ablation) | WFANet · WV3 reduced-resolution | PSNR: 38.486 -> 39.345 | Helped |
| Add SDEM to the network (ablation) | WFANet · WV3 reduced-resolution | PSNR: 38.993 -> 39.345 | Helped |
| Use multi-scale fusion instead of a single-scale network (ablation) | WFANet · WV3 reduced-resolution | PSNR: 38.851 -> 39.345 | Helped |
| Use Frequency Adaptation Blocks instead of Convolution Blocks (ablation) | WFANet SDEM · WV3 reduced-resolution | PSNR: 39.074 -> 39.345 | Helped |
| Use the original Frequency-Query, Spatial-Key, and Fusion-Value role assignment instead of V1 (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 39.092 -> 39.345 | Helped |
| Use the original Frequency-Query, Spatial-Key, and Fusion-Value role assignment instead of V2 (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 38.954 -> 39.345 | Helped |
| Use the original Frequency-Query, Spatial-Key, and Fusion-Value role assignment instead of V3 (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 38.916 -> 39.345 | Helped |
| Use the original Frequency-Query, Spatial-Key, and Fusion-Value role assignment instead of V4 (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 39.056 -> 39.345 | Helped |
| Use the original Frequency-Query, Spatial-Key, and Fusion-Value role assignment instead of V5 (ablation) | WFANet Frequency Attention Triplet · WV3 reduced-resolution | PSNR: 38.573 -> 39.345 | Helped |
Surprising findings
- Ablation shows that removing the MFFA attention entirely drops WV3 PSNR to 38.486 dB, below the spatial-only SDEM-removed variant, indicating the attention fusion is the dominant contributor.
- Swapping roles of the triplet components (e.g., V5) degrades PSNR to 38.573 dB, showing the specific role assignment matters.
Limitations
- Only two scales used in the wavelet pyramid because of dataset size limits.
- Full-resolution results are reported with HQNR and D metrics only on limited datasets, with the paper's own generalization claims based on WV3 and GF2.
- Real-world generalization is claimed but evaluated only through full-resolution assessment.
Experimental setup
Datasets: WorldView-3 (WV3) QuickBird (QB) GaoFen-2 (GF2) PanCollection
Benchmarks: WV3 reduced-resolution QB reduced-resolution GF2 reduced-resolution WV3 full-resolution GF2 full-resolution
Metrics: PSNR SAM ERGAS Q4 Q8 HQNR D_lambda D_s
Baselines: MTF-GLP-FS BDSD-PC TV PNN PanNet DiCNN FusionNet U2Net PanMamba CANNet
Builds on: PanFormer Wave-ViT FAFNet PNN
Closest papers by meaning
- Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion (CVPR 2024 · similarity 0.89)
- FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask (AAAI 2025 · similarity 0.88)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.87)
- Binarized Neural Network for Multi-spectral Image Fusion (CVPR 2025 · similarity 0.86)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.86)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.