Research map / Image generation / Image restoration
Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces two full-Stokes spectro-polarimetric datasets (2,022 trichromatic and 311 hyperspectral images) of real-world scenes and analyzes their statistics, compression, denoising, and spectral dependency of shape-from-polarization.
Problem
Existing spectro-polarimetric image datasets lack real-world scene diversity, illumination variety, circular polarization data, and image count, limiting data-driven spectro-polarimetric vision research.
Approach
Capture two large-scale full-Stokes (linear and circular) datasets, one trichromatic and one hyperspectral (21 bands, 450-650 nm), across diverse indoor/outdoor scenes and illumination, with per-image labels, then use them for statistical analysis and representation learning (PCA and an INR adapted from NeSpoF).
Key results
- A hyperspectral Stokes image (100 MB) is represented by 2.22 MB of PCA coefficients with reconstruction error 2.69e-5 MSE.
- An INR with 8 MLP layers (BPP 60) reaches reconstruction error 1.90e-5, also 2.22 MB of parameters.
- Denoised single-shot capture reaches about 34.5 dB PSNR for Stokes reconstruction; averaging 4/25 shots is needed for 35 dB on trichromatic/hyperspectral cameras.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use INR instead of PCA for hyperspectral Stokes image representation | Hyperspectral Stokes image representation · Hyperspectral Stokes image | MSE: 2.69 × 10−5 -> 1.90×10−5 | Helped |
| Apply pretrained intensity denoising before Stokes reconstruction | Single-shot Stokes imaging · Trichromatic Stokes image | PSNR: 34.5 dB | Helped |
| Increase burst averaging to 4 captures | Trichromatic Stokes camera · Trichromatic Stokes image | PSNR: 35 dB | Helped |
| Increase burst averaging to 25 captures | Hyperspectral Stokes camera · Hyperspectral Stokes image | PSNR: 35 dB | Helped |
| Represent the hyperspectral image with PCA coefficients | Hyperspectral Stokes image representation · Hyperspectral Stokes image | MSE: 2.69 × 10−5 | Helped |
| Represent the hyperspectral image with an 8-layer INR | Hyperspectral Stokes image representation · Hyperspectral Stokes image | MSE: 1.90×10−5 | Helped |
Surprising findings
- Learned intensity denoiser pretrained on RGB images (KBNet) improves Stokes reconstruction despite no polarization training data.
- Normals estimated by a state-of-the-art SfP method vary across spectral channels, with x and y components showing larger variation than z.
Limitations
- Hyperspectral dataset has lower SNR requiring many averaged shots for high PSNR
- Filtering of invalid Stokes vectors (DoP violations, saturated pixels) removes about 1% of data
- Environment labels limited to indoor/outdoor and few illumination classes
Experimental setup
Datasets: Trichromatic Stokes dataset Hyperspectral Stokes dataset
Benchmarks: Trichromatic Stokes dataset Hyperspectral Stokes dataset
Metrics: PSNR MSE BPP standard deviation of normals
Baselines: KBNet Lei et al. SfP method
Builds on: NeSpoF KBNet Lei et al. shape-from-polarization method
Closest papers by meaning
- PolarScale: A Physics-Grounded Benchmark for Radiometrically Consistent RGB-to-Stokes Estimation (NeurIPS 2026 · similarity 0.87)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.87)
- PolarFree: Polarization-based Reflection-Free Imaging (CVPR 2025 · similarity 0.85)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.83)
- SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field (AAAI 2024 · similarity 0.83)
- SPIDeRS: Structured Polarization for Invisible Depth and Reflectance Sensing (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.