Research map / 3D vision / Depth and stereo
FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FMPM-DNet, a dual-stage dynamic network for hyperspectral pansharpening that uses a complex-domain wave-function dynamic feature modulation unit and learnable probability-mask feature reconstruction units.
Problem
Hyperspectral pansharpening suffers from spectral and spatial distortions because static networks ignore spatial variation across HS images and dense connections cause feature redundancy when modeling complex multi-channel mappings.
Approach
Transforms PAN and HS features into complex-domain wave-like features (amplitude and phase via Euler's formula), dynamically generates spatial and spectral modulation matrices to coarsely fuse them (DFMU), then reconstructs fine fused features with learnable global and local Gumbel-softmax probability masks (FPMU).
Key results
- On Pavia center, FMPM-DNet reaches PSNR 38.8193 dB and SAM 4.6083, versus 38.1569 dB for the second-best Tree-SNet.
- On Botswana, PSNR 45.6029 dB, 0.3751 dB above the second-best method; on Chikusei, PSNR 44.0595 dB, 0.8440 dB above the second best.
- Ablation on Pavia center shows the dual-stage structure, DFMU and wave function each improve PSNR, with GPES and LPES masks both contributing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the first stage to a second-stage-only network (ablation) | FMPM-DNet · Pavia center | PSNR: 38.0810 -> 38.8193 | Helped |
| Add the second stage to a first-stage-only network (ablation) | FMPM-DNet · Pavia center | PSNR: 38.4756 -> 38.8193 | Helped |
| Use DFMU instead of concatenating PAN and HS feature maps (ablation) | FMPM-DNet · Pavia center | PSNR: 38.6033 -> 38.8193 | Helped |
| Use the wave function instead of 1×1 convolutions in DFMU (ablation) | FMPM-DNet · Pavia center | PSNR: 38.4758 -> 38.8193 | Helped |
| Add probability masks to GPES and LPES (ablation) | FMPM-DNet · Pavia center | PSNR: 38.4795 -> 38.8193 | Helped |
| Use FMPM-DNet instead of Tree-SNet | HS pansharpening · Pavia center | PSNR: 38.1569 -> 38.8193 | Helped |
| Use FMPM-DNet instead of Tree-SNet | HS pansharpening · Botswana | PSNR: 45.2278 -> 45.6029 | Helped |
| Use FMPM-DNet instead of Tree-SNet | HS pansharpening · Chikusei | PSNR: 43.2155 -> 44.0595 | Helped |
Surprising findings
- Removing the second (FPMU) stage alone gives a PSNR of 38.0810 dB, lower than using the first stage alone (38.4756 dB), and the w/o DFMU variant has higher SSIM (0.9662) than the full model (0.9633).
Limitations
- Parameter count and FLOPs are higher than lightweight baselines such as DIP-Hyperkite and Hyper-DSNet.
- Downstream classification evaluated only on Pavia center with k-means in ENVI, a limited test.
Experimental setup
Datasets: Pavia center Botswana Chikusei
Benchmarks: Pavia center Botswana Chikusei
Metrics: PSNR SAM SSIM SCC RMSE ERGAS
Baselines: CNMF DARN DIP-Hyperkite Hyper-DSNet FPFNet HyperRefiner Tree-SNet
Builds on: Tree-SNet HyperRefiner Gumbel softmax Hyper-DSNet
Closest papers by meaning
- Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening (AAAI 2025 · similarity 0.88)
- Hyperspectral Pansharpening via Diffusion Models with Iteratively Zero-Shot Guidance (CVPR 2025 · similarity 0.86)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.86)
- HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectral Images Fusion (ICML 2025 · similarity 0.85)
- Binarized Neural Network for Multi-spectral Image Fusion (CVPR 2025 · similarity 0.85)
- Learning Multi-Modal Cross-Scale Deformable Transformer Network for Unregistered Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.