Research map / Architectures and efficiency / CNN design
A Selective Re-learning Mechanism for Hyperspectral Fusion Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SRLF-Net, a hyperspectral-multispectral fusion network that uses a Mamba-based preliminary fusion followed by SSG-SRL, which uses the observation model to select only spatially/spectrally distorted feature points for re-learning via Spectral Transformer blocks.
Problem
Hyperspectral fusion (combining low-resolution HSI with MSI to get high-resolution HSI) has high computational cost, and existing methods process all pixels uniformly despite smooth regions needing only shallow processing.
Approach
Extract features from all pixels with a multi-scale Mamba (SS2D) preliminary fusion module, then use the observation model to generate pseudo-MSI and pseudo-LRHSI, compute SSIM/SAM maps, and re-learn only the lowest-quality feature points with Spectral Transformer blocks, thereby reducing FLOPs and overfitting.
Key results
- On CAVE, SRLF-Net reaches 49.3881 dB PSNR with 1.33M params and 81.22G FLOPs, versus 48.5910 dB for the next best method Mog-DCN (4.390T FLOPs).
- On Harvard, SRLF-Net achieves 47.8954 dB PSNR with SAM 2.6876, slightly above Mog-DCN's 47.8910 dB.
- On Gaofen5 no-reference QNR, SRLF-Net scores 0.9733, the highest among compared methods (next best DSPNet 0.9722).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SRLF-Net instead of Mog-DCN | Hyperspectral fusion · CAVE | PSNR: 48.5910 -> 49.3881 | Helped |
| Use SRLF-Net instead of Mog-DCN | Hyperspectral fusion · CAVE | SAM: 2.2541 -> 2.1816 | Helped |
| Use SRLF-Net instead of Mog-DCN | Hyperspectral fusion · Harvard | PSNR: 47.8910 -> 47.8954 | Helped |
| Use SRLF-Net instead of Mog-DCN | Hyperspectral fusion · Harvard | UIQI: 0.8961 -> 0.8956 | Hurt |
| Use SRLF-Net instead of Mog-DCN | Hyperspectral fusion · Gaofen5 | QNR: 0.9720 -> 0.9733 | Helped |
| Increase network depth from N = 1 to N = 2 (ablation) | SRLF-Net · CAVE | PSNR: 48.2703 -> 48.7428 | Helped |
| Increase network depth from N = 2 to N = 3 (ablation) | SRLF-Net · CAVE | PSNR: 48.7428 -> 49.0562 | Helped |
| Increase network depth from N = 3 to N = 4 (ablation) | SRLF-Net · CAVE | PSNR: 49.0562 -> 49.3881 | Helped |
| Increase network depth from N = 4 to N = 5 (ablation) | SRLF-Net · CAVE | PSNR: 49.3881 -> 49.5214 | Helped |
| Add the multi-scale structure to the Preliminary Fusion Module (ablation) | SRLF-Net · CAVE | PSNR: 48.0473 -> 49.3881 | Helped |
| Add SSG-SRL selection for targeted re-learning (ablation) | SRLF-Net · CAVE | PSNR: 49.3135 -> 49.3881 | Helped |
| Use known observation-model response functions instead of learned layers (ablation) | SRLF-Net · CAVE | PSNR: 49.3412 -> 49.3881 | Helped |
Surprising findings
- Removing SSG-SRL slightly lowers PSNR (49.3135 vs 49.3881 dB) yet increases FLOPs by 5.43G, indicating selection mainly saves compute.
- Replacing the observation-model prior R and C with learned layers yields nearly the same PSNR (49.3412 dB), suggesting the prior is not critical.
Limitations
- Selection ratio lambda must be tuned; SAM fluctuates for lambda above 0.3 and rises beyond 0.5.
- Real-data evaluation relies on no-reference QNR since no ground-truth HRHSI exists for Gaofen5.
- Relies on known spectral and spatial response functions, assumed known in the observation model.
Experimental setup
Datasets: CAVE Harvard Gaofen5
Benchmarks: CAVE Harvard Gaofen5
Metrics: PSNR SAM UIQI SSIM QNR FLOPs Params
Baselines: Hysure NSSR DAEM FusionMamba DHIF-Net DSPNet MIMO-SST Mog-DCN LRTN
Builds on: SS2D (VMamba) Mamba FusionMamba Spectral Transformer
Closest papers by meaning
- Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion (NeurIPS 2024 · similarity 0.85)
- OTPNet: ODE-inspired Tuning-free Proximal Network for Remote Sensing Image Fusion (AAAI 2025 · similarity 0.85)
- HetSSNet: Spatial-Spectral Heterogeneous Graph Learning Network for Panchromatic and Multispectral Images Fusion (ICML 2025 · similarity 0.85)
- SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction (CVPR 2024 · similarity 0.85)
- SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening (NeurIPS 2024 · similarity 0.85)
- Self-Learning Hyperspectral and Multispectral Image Fusion via Adaptive Residual Guided Subspace Diffusion Model (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.