Research map / Image generation / Image restoration
Frequency-Adaptive Pan-Sharpening with Mixture of Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FAME, a pan-sharpening network that predicts a DCT-based frequency mask with Gumbel-Softmax, routes low- and high-frequency features through separate mixture-of-experts modules, and fuses them with a sparse top-k expert mixture.
Problem
Pan-sharpening must reconstruct lost high-frequency detail in low-resolution multispectral images guided by a high-resolution panchromatic image, but existing CNN and transformer methods apply uniform processing across frequency components and handle diverse remote sensing content poorly.
Approach
Learn the frequency separation mask adaptively (instead of a hand-thresholded DCT mask) and use separate low- and high-frequency mixture-of-experts branches plus a top-k expert-mixture fusion module so the network dynamically adapts to content.
Key results
- On WV2 reduced-resolution test, FAME reaches PSNR 42.0262 and SAM 0.0215, above SFINet (41.7244 PSNR).
- On WV3, PSNR improves by about 0.4 dB over INNformer (30.9903 vs 30.5365).
- On full-resolution GaoFen2, FAME achieves the best QNR of 0.8291 among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FAME-Net instead of SFINet | Pan-sharpening · WorldView-II | PSNR: 41.7244 -> 42.0262 | Helped |
| Use FAME-Net instead of SFINet | Pan-sharpening · GaoFen2 | PSNR: 47.4712 -> 47.6721 | Helped |
| Use FAME-Net instead of SFINet | Pan-sharpening · WorldView-III | PSNR: 30.5901 -> 30.9903 | Helped |
| Add the mask predictor and frequency split (ablation) | FAME-Net · WorldView-II | PSNR: 41.8998 -> 42.0261 | Helped |
| Add the mask predictor and frequency split (ablation) | FAME-Net · GaoFen2 | PSNR: 47.5914 -> 47.6721 | Helped |
| Add the mask predictor and frequency split (ablation) | FAME-Net · WorldView-III | PSNR: 30.8387 -> 30.9903 | Helped |
| Use the Experts Mixture module instead of a ResBlock (ablation) | FAME-Net · WorldView-II | PSNR: 41.8274 -> 42.0261 | Helped |
| Use the Experts Mixture module instead of a ResBlock (ablation) | FAME-Net · GaoFen2 | PSNR: 47.3596 -> 47.6721 | Helped |
| Use the Experts Mixture module instead of a ResBlock (ablation) | FAME-Net · WorldView-III | PSNR: 30.7930 -> 30.9903 | Helped |
Limitations
- Frequency mask is supervised by a manually thresholded DCT mask label, which the method itself notes is threshold-dependent.
- Evaluation limited to three satellite datasets and full-resolution evaluation uses only no-reference metrics on one dataset.
Experimental setup
Datasets: WorldView-II GaoFen2 WorldView-III
Benchmarks: WorldView-II GaoFen2 WorldView-III
Metrics: PSNR SSIM SAM ERGAS D_lambda D_s QNR
Baselines: PANNET MSDCNN SRPPNN GPPNN MutNet INNformer SFINet PNN GFPCA GS IHS Brovey
Builds on: Half-Instance Normalization Gumbel-Softmax mixture of experts PANNET SFINet
Closest papers by meaning
- Progressive High-Frequency Reconstruction for Pan-Sharpening with Implicit Neural Representation (AAAI 2024 · similarity 0.88)
- Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion (CVPR 2024 · similarity 0.87)
- Accelerated Diffusion via High-Low Frequency Decomposition for Pan-Sharpening (AAAI 2025 · similarity 0.86)
- Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration (AAAI 2024 · similarity 0.86)
- Binarized Neural Network for Multi-spectral Image Fusion (CVPR 2025 · similarity 0.86)
- Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.