Research map / Image generation / Super-resolution
EigenSR: Eigenimage-Bridged Pre-Trained RGB Learners for Single Hyperspectral Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Transfers a pre-trained RGB transformer (IPT) to single hyperspectral image super-resolution by fine-tuning a single-channel model on SVD eigenimages and inferring with an iterative spectral regularization that re-estimates the spectral basis.
Problem
Single hyperspectral image super-resolution is bottlenecked by scarce HSI training data, and pre-trained RGB SR models cannot directly handle the many spectral channels of HSI.
Approach
Decompose HSI via SVD into spectral basis and spatial eigenimages; fine-tune a single-channel pre-trained RGB transformer (IPT, with LoRA and Head/Tail) on eigenimages, then at inference super-resolve the top-R eigenimages channel-by-channel and recombine with the spectral basis, iteratively refining the basis with a weighted average (ISR).
Key results
- EigenSR-β reaches 40.46 dB PSNR and 1.182 SAM on ARAD_1K-Test at ×4, versus 39.96 dB for the IPT baseline and 40.40 dB for ESSAformer.
- EigenSR-α runs in 1.13 s versus 3.19 s for the IPT baseline on ARAD_1K ×4 while improving PSNR to 40.28 dB.
- In the blind ×4 remote-sensing test, EigenSR-α has the lowest NRQMS on Chikusei (20.58) and DC Mall (18.05) among the methods listed, with inference times of 3.36 s and 4.76 s respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EigenSR-β instead of the strongest PSNR baseline | Single-HSI-SR · ARAD_1K-Test, CAVE, and Harvard at ×2, ×4, and ×8 | PSNR: 49.09 -> 49.11; 44.49 -> 44.97; 45.56 -> 45.77; 40.40 -> 40.46; 37.77 -> 38.36; 40.04 -> 40.21; 34.71 -> 34.75; 33.00 -> | Helped |
| Use EigenSR-β instead of the strongest baseline | Single-HSI-SR blind testing · Pavia, DC Mall, and Chikusei ×4 | NRQMS: 19.19 -> 19.13; 18.33 -> 17.98; 20.72 -> 20.42 | Helped |
| Add eigenimage-domain fine-tuning to the IPT baseline (ablation) | IPT baseline · ARAD_1K-Test ×4 | PSNR: 39.96 -> 40.28 | Helped |
| Add NMF spectral constraint (ablation) | IPT baseline · ARAD_1K-Test ×4 | PSNR: 39.96 -> 39.81 | Hurt |
| Add IPT pre-trained weights instead of training from scratch (ablation) | EigenSR-α · ARAD_1K-Test ×4 | PSNR: 39.74 -> 40.28 | Helped |
| Add iterative spectral regularization (ablation) | EigenSR-α · ARAD_1K-Test ×4 | PSNR: 40.28 -> 40.46 | Helped |
| Fine-tune Head & Tail plus LoRA instead of only Head & Tail (ablation) | EigenSR · ARAD_1K-Test ×4 | PSNR: 39.92 -> 40.46 | Helped |
| Fine-tune all parameters instead of Head & Tail plus LoRA (ablation) | EigenSR · ARAD_1K-Test ×4 | PSNR: 40.46 -> 40.60 | Helped |
Surprising findings
- NMF spectral constraint from prior RGB-transfer work lowers performance relative to the IPT baseline (39.81 vs 39.96 dB) while adding large inference time.
- Full-parameter fine-tuning is only marginally better than Head & Tail + LoRA (40.60 vs 40.46 dB) but uses about 116M vs 1.11M trainable parameters.
Limitations
- ISR adds inference cost (about 4.7 s vs 1.1 s on ARAD_1K ×4) and requires tuning λ per scale.
- Reliance on RGB-pretrained IPT; the single-channel eigenimage domain needed the first eigenimage only for RGB fine-tuning, indicating RGB eigenimages carry little signal beyond the first component.
- Blind remote-sensing evaluation uses no-reference NRQMS only, and several baselines are not best on all datasets.
Experimental setup
Datasets: ARAD_1K RESISC45
Benchmarks: ARAD_1K-Test CAVE Harvard Pavia DC Mall Chikusei
Metrics: PSNR SSIM SAM NRQMS inference time
Baselines: Bicubic DHSP DKP SSPSR SFCSR RFSR TSBSR MSDformer ESSAformer
Builds on: IPT LoRA TSBSR
Closest papers by meaning
- S2CycleDiff: Spatial-Spectral-Bilateral Cycle-Diffusion Framework for Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.88)
- Learning Multi-Modal Cross-Scale Deformable Transformer Network for Unregistered Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.88)
- Enhancing Hyperspectral Images via Diffusion Model and Group-Autoencoder Super-resolution Network (AAAI 2024 · similarity 0.87)
- Automatic Spectral Calibration of Hyperspectral Images: Method, Dataset and Benchmark (CVPR 2025 · similarity 0.87)
- Hyperspectral Pansharpening via Diffusion Models with Iteratively Zero-Shot Guidance (CVPR 2025 · similarity 0.87)
- HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.