Research map / 3D vision / 3D generation
Sp3ctralMamba: Physics-Driven Joint State Space Model for Hyperspectral Image Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sp3ctralMamba, a joint state space model for snapshot compressive hyperspectral reconstruction that combines spatial/frequency Mamba scans with three physics priors (mask consistency, energy, structure).
Problem
Hyperspectral image reconstruction from 2D coded snapshot compressive measurements (CASSI), where spectral information is spatially sparse and CNN/ViT methods struggle with long-range dependencies or cost.
Approach
Joint SSM block S3Mamba that runs non-local vanilla scanning, local patch scanning, and a spiral scan over Fourier amplitude/phase in parallel per band group, plus physics-driven losses (mask-based imaging consistency, Retinex-style energy prior, gradient-based structural prior) applied with stop-gradient staging.
Key results
- Average PSNR 41.21 dB and SSIM 0.989 on 10 KAIST simulation scenes, outperforming the second-best (SPECAT, 40.37 dB) by about 0.8 dB.
- Ablation: replacing LSSM and FSSM with a single vanilla scan drops PSNR from 41.21 to 38.51 dB; full S3MAB with both scans reaches 41.21 dB vs 38.51 without them.
- Physics priors: SPC adds 1.1 dB and IPCC adds 0.4 dB over the no-prior baseline (39.42 to 40.58 dB with SPC alone; full model 41.21 dB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Sp3ctralMamba instead of SPECAT | HSI reconstruction · KAIST (10 scenes, Avg) | PSNR: 40.37 -> 41.21 | Helped |
| Use Sp3ctralMamba instead of SPECAT | HSI reconstruction · KAIST (10 scenes, Avg) | SSIM: 0.986 -> 0.989 | Helped |
| Add LSSM and FSSM rather than using NLSSM alone (ablation) | S3Mamba · KAIST (simulated) | PSNR: 38.51 -> 41.21 | Helped |
| Add LSSM to the NLSSM-based variant (ablation) | S3Mamba · KAIST (simulated) | PSNR: +1.7dB | Helped |
| Add FSSM to the NLSSM-based variant (ablation) | S3Mamba · KAIST (simulated) | PSNR: +1.3dB | Helped |
| Add IPCC mask modulation (ablation) | Sp3ctralMamba · KAIST (simulated) | PSNR: 39.42 -> 39.90 | Helped |
| Add EPC to constrain encoded-feature pixel intensity (ablation) | Sp3ctralMamba · KAIST (simulated) | PSNR: 39.42 -> 40.21 | Helped |
| Add SPC to enhance high-frequency signals in decoded features (ablation) | Sp3ctralMamba · KAIST (simulated) | PSNR: 39.42 -> 40.58 | Helped |
Surprising findings
- The end-to-end Mamba model without physical constraints already nearly reaches SOTA, suggesting Mamba's spatial modeling is strong on its own.
- Structural prior (SPC) gave more gain than the energy prior (EPC), indicating loss of high-frequency structure in decoder features is a key bottleneck.
Limitations
- Real-data evaluation has no ground truth, so only visual comparison with RGB references is possible.
- Reported GFLOPs (64.65) and parameters (0.45M) are higher in FLOPs than SPECAT, so the efficiency claim is relative to some baselines only.
Experimental setup
Datasets: CAVE KAIST
Benchmarks: KAIST (10 scenes) real CASSI measurements (5 scenes)
Metrics: PSNR SSIM Params GFLOPs
Baselines: TSA-Net GAP-net MST-L MST++ CST-L DAUHST-9stg PADUT-12stg SST-LPlus SPECAT
Builds on: Mamba VMamba MST Retinex
Closest papers by meaning
- Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging (AAAI 2025 · similarity 0.89)
- Improving Spectral Snapshot Reconstruction with Spectral-Spatial Rectification (CVPR 2024 · similarity 0.86)
- SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction (CVPR 2024 · similarity 0.86)
- Dual-Window Multiscale Transformer for Hyperspectral Snapshot Compressive Imaging (AAAI 2024 · similarity 0.86)
- EigenSR: Eigenimage-Bridged Pre-Trained RGB Learners for Single Hyperspectral Image Super-Resolution (AAAI 2025 · similarity 0.85)
- Learning Multi-Modal Cross-Scale Deformable Transformer Network for Unregistered Hyperspectral Image Super-resolution (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.