Research map / Image generation / Image restoration
SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPECAT, a hierarchical spatial-spectral cumulative-attention transformer with a dual-domain (object + projection) loss for compressive hyperspectral image reconstruction from snapshot systems.
Problem
Compressive spectral image reconstruction from snapshot measurements struggles to capture non-local spatial-spectral mutation features, and current SOTA methods gain accuracy only by greatly increasing depth and parameters.
Approach
Cumulative-Attention Blocks chain a windowed spatial attention (with a mask attention term) into a spectral-wise attention so that lower-level spatial attention highlights mutation regions for upper-level spectral attention, combined with a loss that enforces both the object-domain and the projection-domain (measurement) constraints.
Key results
- SPECAT achieves 40.37 dB average PSNR and 0.986 SSIM on 10 KAIST simulation scenes, versus 39.13 dB for the best prior method (SST-LPlus) and 38.96 dB for PADUT-12stg.
- SPECAT uses 0.29M parameters and 12.4 GFLOPs, about 5% of the parameters and 10% of the compute of the SOTA methods they compare against.
- Ablation: adding CAB raises PSNR from 34.02 to 38.87 dB, and the dual-domain loss adds about 0.4 dB (39.96 to 40.37 dB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SPECAT instead of SST-LPlus | Optical filter-based HSI reconstruction · KAIST 10-scene average | PSNR: 39.13 -> 40.37 | Helped |
| Add HSA to the spectral-wise Transformer baseline (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 34.02 -> 38.87 | Helped |
| Add MA to the HSA configuration (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 38.87 -> 39.96 | Helped |
| Add DLF to the HSA and MA configuration (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 39.96 -> 40.37 | Helped |
| Increase CAB configuration from [2, 1] to [2, 2] (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 40.37 -> 40.42 | Helped |
| Increase depth and use CAB configuration [2, 2, 1] (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 40.37 -> 40.68 | Helped |
| Increase depth and use CAB configuration [2, 2, 2] (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | PSNR: 40.37 -> 40.70 | Helped |
| Increase CAB count and network depth (ablation) | SPECAT · Optical filter-based HSI, KAIST 10-scene average | GFLOPs: 12.4 -> 13.9; 19.9; 21.5 | Hurt |
Surprising findings
- Increasing CAB depth or attention modules (e.g., [2,2,2] config) gives only marginal PSNR gains (40.70 dB) while greatly increasing parameters, supporting the claim of limited returns from scaling.
Limitations
- Performance on the real CASSI system data shows room for improvement in spatial detail recovery, per the authors.
- Gains from increasing network depth or attention modules are limited, so scaling does not yield much benefit.
Experimental setup
Datasets: CAVE KAIST TSA-Net real CASSI dataset
Benchmarks: KAIST (10 test scenes, optical filter-based simulation) CASSI real measurements (TSA-Net dataset)
Metrics: PSNR SSIM Params GFLOPs
Baselines: TwIST DeSCI TSA-Net GAP-net BIRNAT MST-L MST++ CST-L DAUHST-9stg PADUT-12stg SST-LPlus
Builds on: MST Swin Transformer SST hierarchical attention network U-Net
Closest papers by meaning
- Improving Spectral Snapshot Reconstruction with Spectral-Spatial Rectification (CVPR 2024 · similarity 0.89)
- Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues (AAAI 2024 · similarity 0.88)
- Spectral Compressive Imaging via Chromaticity-Intensity Decomposition (NeurIPS 2025 · similarity 0.88)
- Dual-Window Multiscale Transformer for Hyperspectral Snapshot Compressive Imaging (AAAI 2024 · similarity 0.87)
- LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.87)
- VolFormer: Explore More Comprehensive Cube Interaction for Hyperspectral Image Restoration and Beyond (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.