Research map / Image generation / Image restoration
Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FeINFN, an implicit neural representation network that fuses LR-HSI and HR-MSI in both spatial and Fourier domains, with a Gabor-wavelet-activated spatial-frequency decoder, reaching SOTA on CAVE and Harvard ×4.
Problem
Multispectral and hyperspectral image fusion (MHIF) to produce high-resolution hyperspectral images; existing INR-based fusion loses high-frequency information and lacks global receptive field.
Approach
Transforms INR latent codes into the Fourier domain, handling amplitude and phase separately with implicit interpolation fusion, combined with spatial-domain INR fusion; a decoder using a complex Gabor wavelet activation (claimed time-frequency tight) fuses spatial and frequency features.
Key results
- On CAVE ×4, FeINFN reaches PSNR 52.47 dB, SAM 1.91, ERGAS 0.98, SSIM 0.998 with 3.165M params, best across metrics.
- On Harvard ×4, FeINFN reaches PSNR 49.06 dB and ERGAS 1.78, improving PSNR by 0.14 dB over the prior best (BDT).
- Ablation: removing the Fourier domain drops CAVE ×4 PSNR from 52.47 to 47.86 dB; spatial-only gives 52.11 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FeINFN instead of BDT | MHIF · CAVE ×4 | PSNR: 52.30±3.98 -> 52.47±4.10 | Helped |
| Use FeINFN instead of BDT | MHIF · CAVE ×4 | SAM: 1.93±0.55 -> 1.91±0.59 | Helped |
| Use FeINFN instead of BDT | MHIF · CAVE ×4 | ERGAS: 1.02±0.77 -> 0.98±0.74 | Helped |
| Use FeINFN instead of BDT | MHIF · Harvard ×4 | PSNR: 48.83±3.45 -> 49.06±3.15 | Helped |
| Use FeINFN instead of BDT | MHIF · Harvard ×4 | ERGAS: 1.83±0.81 -> 1.78±0.75 | Helped |
| Use FeINFN instead of BDT | MHIF · Harvard ×4 | SAM: 2.07±0.49 -> 2.10±0.53 | Hurt |
| Replace INR with Pixel Shuffle (ablation) | FeINFN · CAVE ×4 | PSNR: 52.26±4.37 -> 52.47±4.10 | Hurt |
| Remove the Fourier domain branch (ablation) | FeINFN · CAVE ×4 | PSNR: 52.11±4.22 -> 52.47±4.10 | Hurt |
| Remove the spatial domain branch (ablation) | FeINFN · CAVE ×4 | PSNR: 47.86±3.42 -> 52.47±4.10 | Hurt |
| Use complex Gabor wavelet activation instead of ReLU (ablation) | FeINFN decoder · CAVE ×4 | PSNR: 52.03±3.84 -> 52.47±4.10 | Helped |
| Use complex Gabor wavelet activation instead of GELU (ablation) | FeINFN decoder · CAVE ×4 | PSNR: 51.96±3.88 -> 52.47±4.10 | Helped |
| Use complex Gabor wavelet activation instead of Leaky ReLU (ablation) | FeINFN decoder · CAVE ×4 | PSNR: 51.98±3.92 -> 52.47±4.10 | Helped |
Surprising findings
- Fourier amplitudes of HR-HSI and LR-HSI latent codes are very similar while phases differ, motivating separate amplitude/phase handling.
- Fourier-domain-only branch performs much worse (47.86 dB PSNR) than spatial-only (52.11 dB) on CAVE ×4.
Limitations
- Code is only promised on an anonymous GitHub after acceptance; no released artifacts.
- Evaluated only on two simulated benchmark datasets (CAVE and Harvard) at limited scales.
Experimental setup
Datasets: CAVE Harvard
Benchmarks: CAVE Harvard
Metrics: PSNR SAM ERGAS SSIM
Baselines: Bicubic CSTF-FUS LTTR LTMR IR-TenSR ResTFNet SSRNet HSRNet MoGDCN Fusformer DHIF PSRT
Builds on: LIIF WIRE EDSR GFNet FDA
Closest papers by meaning
- Learning Spatially Collaged Fourier Bases for Implicit Neural Representation (AAAI 2024 · similarity 0.86)
- OTIAS: OcTree Implicit Adaptive Sampling for Multispectral and Hyperspectral Image Fusion (AAAI 2025 · similarity 0.86)
- A Selective Re-learning Mechanism for Hyperspectral Fusion Imaging (CVPR 2025 · similarity 0.85)
- FINER: Flexible Spectral-bias Tuning in Implicit NEural Representation by Variable-periodic Activation Functions (CVPR 2024 · similarity 0.85)
- Implicit Neural Representations and the Algebra of Complex Wavelets (ICLR 2024 · similarity 0.84)
- Learning Multi-Modal Cross-Scale Deformable Transformer Network for Unregistered Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.