Research map / Image generation / Image restoration
Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CESST, a transformer-based RGB-to-hyperspectral reconstruction network that processes R, G, B channels independently and fuses them via a spectrum-fusion attention module with six cross-channel branches, plus a spatio-spectral attention block.
Problem
Learning-based RGB-to-hyperspectral image reconstruction underexploits inter-band information, especially in long-wavelength bands, because existing methods project RGB channels jointly into the spectral space early on.
Approach
Model each RGB channel independently in a high-dimensional embedding space before fusion; query pairwise cross-channel correlations combinatorially with six parallel spectral-MSA branches (SFAM); and extract spatial and spectral self-attention in parallel with a shuffle-window spatial MSA (SSAB).
Key results
- On NTIRE2022 HSI, CESST reports PSNR 35.14 dB, MRAE 0.1497 and RMSE 0.0225, the best of all compared methods, with 1.54M params.
- On ICVL, CESST achieves PSNR 33.25 dB, versus 31.41 dB for MST++ (the best prior method in the table).
- Ablation: independent modeling adds 3.69 dB PSNR over the baseline; SSAB adds 1.88 dB and SFAM adds 1.43 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add independent modeling of RGB channels (ablation) | CESST baseline · NTIRE2022 HSI dataset | PSNR: 28.14 -> 31.83 | Helped |
| Add SSAB (ablation) | CESST with independent modeling · NTIRE2022 HSI dataset | PSNR: 31.83 -> 33.71 | Helped |
| Add SFAM (ablation) | CESST with independent modeling and SSAB · NTIRE2022 HSI dataset | PSNR: 33.71 -> 35.14 | Helped |
| Replace concatenation-convolution with SFAM (ablation) | CESST fusion scheme · NTIRE2022 HSI dataset | PSNR: 33.71 -> 35.14; 1.43dB | Helped |
| Replace SFT with SFAM (ablation) | CESST fusion scheme · NTIRE2022 HSI dataset | PSNR: 34.22 -> 35.14; 0.92dB | Helped |
| Replace EFF with SFAM (ablation) | CESST fusion scheme · NTIRE2022 HSI dataset | PSNR: 34.73 -> 35.14; 0.41dB | Helped |
| Use spatio-spectral MSA instead of spatial-MSA (ablation) | CESST attention block · NTIRE2022 HSI dataset | PSNR: 34.75 -> 35.14 | Helped |
| Use spatio-spectral MSA instead of shifted-window MSA (ablation) | CESST attention block · NTIRE2022 HSI dataset | PSNR: 33.92 -> 35.14 | Helped |
| Use spatio-spectral MSA instead of spectral-MSA (ablation) | CESST attention block · NTIRE2022 HSI dataset | PSNR: 34.96 -> 35.14 | Helped |
| Add spectral-MSA to shuffle-window MSA (ablation) | CESST attention block · NTIRE2022 HSI dataset | PSNR: 34.51 -> 35.14 | Helped |
Surprising findings
- The paper notes its FLOPs (90.18G) exceed MST++ (23.05G) but inference time is comparable on the same GPU.
Limitations
- FLOPs (90.18G) are higher than MST++ (23.05G) on NTIRE2022.
- ICVL lacks aligned RGB images, so RGB inputs are synthesized via a simulation method.
Experimental setup
Datasets: NTIRE2022 HSI dataset ICVL HSI dataset
Benchmarks: NTIRE2022 HSI dataset ICVL HSI dataset
Metrics: MRAE RMSE PSNR ERGAS SAM
Baselines: HSCNN+ HRNet AWAN MST++ EDSR HDNet HINet MIRNet Restormer MPRNet MST-L
Builds on: MST++ UNet Swin Transformer Restormer coordinate attention
Closest papers by meaning
- SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction (CVPR 2024 · similarity 0.88)
- LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.88)
- CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis (ICLR 2026 · similarity 0.87)
- FRN: Fractal-Based Recursive Spectral Reconstruction Network (NeurIPS 2025 · similarity 0.86)
- Improving Spectral Snapshot Reconstruction with Spectral-Spatial Rectification (CVPR 2024 · similarity 0.85)
- Sp3ctralMamba: Physics-Driven Joint State Space Model for Hyperspectral Image Reconstruction (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.