Research map / Image generation / Image restoration
VolFormer: Explore More Comprehensive Cube Interaction for Hyperspectral Image Restoration and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VolFormer, a Transformer for hyperspectral image restoration built on volumetric self-attention (VolSA) that models 3D cube interactions via decomposed spectral and spatial similarity matrices with a spectral location prior.
Problem
Existing Transformer self-attention for hyperspectral image restoration (channel or spatial self-attention) only models interactions in one or two flat dimensions, lacking joint spectral-spatial interaction across the 3D HSI cube, while full 3D attention is computationally cubic.
Approach
Volumetric self-attention composes a spectral-wise projection matrix (channel cross-covariance with a bidirectional exponential decay spectral location prior) and a spatial-wise projection matrix, relayed to approximate 3D cube attention with quadratic complexity.
Key results
- On CAVE x4 SR, VolFormer reaches MPSNR 40.205 vs 40.073 for DSTrans, with PSNR gains of about 0.13 dB over the previous best method.
- On HU2013 classification, VolFormer reaches OA 97.35% vs 96.76% for the best prior baseline (BS2T).
- Ablation on CAVE x4: VolSA achieves MPSNR 40.205 and SAM 3.083, outperforming SSA+CSA (40.073) at 90.15 vs 93.60 GFLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VolFormer instead of DSTrans | HSI super-resolution · CAVE ×4 | MPSNR: 40.073 -> 40.205 | Helped |
| use VolFormer instead of ESSA | HSI super-resolution · Pavia ×4 | MPSNR: 32.806 -> 32.925 | Helped |
| use VolFormer instead of ESSA | HSI super-resolution · Chikusei ×4 | MPSNR: 40.320 -> 40.407 | Helped |
| use VolFormer instead of SST | HSI denoising · ICVL Gaussian noise σ=30 | MPSNR: 43.322 -> 43.551 | Helped |
| use VolFormer instead of SST | HSI denoising · ICVL mixture noise | MPSNR: 36.065 -> 36.213 | Helped |
| use VolFormer instead of SST | HSI denoising · HSIDwRD | MPSNR: 27.542 -> 27.825 | Helped |
| replace HybridFormer spatial-spectral attention with VolSA | HybridFormer classifier · HU2013 | OA: 96.40±0.30 -> 97.35±0.30 | Helped |
| add volumetric self-attention instead of spatial self-attention (ablation) | VolFormer · CAVE ×4 | MPSNR: 39.597 -> 40.205 | Helped |
| add volumetric self-attention instead of channel self-attention (ablation) | VolFormer · CAVE ×4 | MPSNR: 39.647 -> 40.205 | Helped |
| replace combined spatial and channel self-attention with volumetric self-attention (ablation) | VolFormer · CAVE ×4 | MPSNR: 40.073 -> 40.205 | Helped |
| add the spectrum location prior (SLP) (ablation) | VolSA · CAVE ×4 | MPSNR: 40.154 -> 40.205 | Helped |
Limitations
- Trained on heterogeneous data with auxiliary RGB datasets, so gains may depend on that setup
- Spectral location prior adds parameters with an exponential decay assumption on wavelength correlation
- Gains on some classification classes are mixed; e.g., lower accuracy on some classes than baselines
Experimental setup
Datasets: CAVE Pavia Chikusei DIV2K ICVL HSIDwRD RENOIR HU2013
Benchmarks: CAVE Pavia Chikusei ICVL HSIDwRD HU2013
Metrics: SAM CC ERGAS RMSE MPSNR MSSIM OA AA Kappa FLOPs
Baselines: GDRRN MCNet ERCSR SSPSR HSISR DSTrans ESSA BM4D KBR WLRTR NGmeet HSID-CNN
Builds on: Restormer SwinIR retention mechanism HybridFormer DSTrans
Closest papers by meaning
- SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction (CVPR 2024 · similarity 0.86)
- LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift (NeurIPS 2026 · similarity 0.85)
- Dual-Window Multiscale Transformer for Hyperspectral Snapshot Compressive Imaging (AAAI 2024 · similarity 0.85)
- Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration (CVPR 2024 · similarity 0.84)
- Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues (AAAI 2024 · similarity 0.84)
- Improving Spectral Snapshot Reconstruction with Spectral-Spatial Rectification (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.