Research map / Image generation / Image restoration
Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HFIN, a hierarchical network that fuses PAN and LRMS features via spatial, global Fourier, and local (overlapping-region) Fourier branches for pan-sharpening.
Problem
Pan-sharpening fuses low-resolution multispectral and panchromatic images; prior Fourier-based methods only model global frequency relations and ignore local frequency relationships between PAN and LRMS.
Approach
Hierarchically decompose PAN and LRMS features into spatial, global Fourier (whole-image DFT) and local Fourier (50%-overlapping region DFT) information, fuse each independently, then integrate them via spatial-frequency and global-local fusion modules in SGLI blocks.
Key results
- On WorldView-II reduced-resolution, achieves 42.2319 dB PSNR, 0.4 dB above the second-best PanFlowNet (41.8584 dB).
- On GaoFen2, achieves 48.8783 dB PSNR, about 1.0 dB above the second-best SRPPNN (47.1998 dB).
- Ablation on WV2 shows removing the local Fourier block drops PSNR from 42.23 to 40.81 dB at equal parameter count.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add local Fourier information by partitioning regions (ablation) | HFIN · WorldView-II | PSNR: 42.0354 -> 42.2319 | Helped |
| Add spatial-frequency (SF) fusion (ablation) | HFIN · WorldView-II | PSNR: 40.8078 -> 42.2319 | Helped |
| Add global-local (GL) fusion (ablation) | HFIN · WorldView-II | PSNR: 42.0655 -> 42.2319 | Helped |
| Use 50% overlap when partitioning four regions (ablation) | HFIN · WorldView-II | PSNR | Helped |
| Increase the number of local Fourier regions (ablation) | HFIN · WorldView-II | PSNR | Helped |
| Use HFIN instead of PanFlowNet | Pan-sharpening · WorldView-II | SSIM: 0.9712 -> 0.9714 | Helped |
| Use HFIN instead of PanFlowNet | Pan-sharpening · WorldView-III | SSIM: 0.9221 -> 0.9203 | Hurt |
Limitations
- Local Fourier block uses only four regions, since more regions give small gains at large parameter cost.
- Evaluated mainly on reduced-resolution Wald-protocol data; full-resolution evaluation is only no-reference on GaoFen2.
Experimental setup
Datasets: WorldView-II GaoFen2 WorldView-III RoadScene NYU v2
Benchmarks: WorldView-II GaoFen2 WorldView-III GaoFen2 full-resolution real-world RoadScene NYU v2
Metrics: PSNR SSIM SAM ERGAS Ds Dλ QNR MI VIF FMI
Baselines: PNN PANNET MSDCNN SRPPNN GPPNN SFIINET PanFlowNet SFIM Brovey GS IHS GFPCA
Builds on: SFIINet GPPNN PanFlowNet
Closest papers by meaning
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.89)
- Frequency-Adaptive Pan-Sharpening with Mixture of Experts (AAAI 2024 · similarity 0.87)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.86)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.85)
- Probing Synergistic High-Order Interaction in Infrared and Visible Image Fusion (CVPR 2024 · similarity 0.85)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.