Research map / 3D vision / Depth and stereo
Super-Resolution Reconstruction from Bayer-Pattern Spike Streams
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSCSR, a network that super-resolves color images from low-resolution Bayer-pattern color spike camera streams using a spike representation, motion-guided temporal-pixel search, residual feature refinement, and color-correlated reconstruction, trained on a simulator-generated dataset.
Problem
Color spike cameras with Bayer CFA capture high-speed color scenes but have limited spatial resolution; reconstructing high-resolution color images from binary Bayer-pattern spike streams is unexplored.
Approach
Represent Bayer-pattern spike streams with local temporal windows plus global perception, then search temporal pixels per color channel using CFA layout and sub-pixel motion jointly estimated across channels, refine them via residuals to a temporal average, and fuse multi-stage color features for reconstruction.
Key results
- On REDS x2, CSCSR reaches 33.39 dB PSNR and 0.9121 SSIM, versus 32.38 dB for SpikeSR-Net.
- On REDS x4, CSCSR reaches 28.77 dB PSNR, versus 28.28 dB for SpikeSR-Net and 27.56 dB for VidarSR.
- Ablation shows removing the BSSR module drops x2 PSNR from 33.39 to 33.04 dB, and the assembled motion estimation drops it to 30.24 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · REDS-based dataset ×2 | PSNR: 32.38dB -> 33.39dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · Vid4-based dataset ×2 | PSNR: 28.34dB -> 28.77dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · REDS-based dataset ×3 | PSNR: 29.79dB -> 29.92dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · Vid4-based dataset ×3 | PSNR: 25.33dB -> 25.68dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · REDS-based dataset ×4 | PSNR: 28.28dB -> 28.77dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · Vid4-based dataset ×4 | PSNR: 23.59dB -> 23.79dB | Helped |
| use CSCSR instead of the strongest baseline, SpikeSR-Net | Color spike camera super-resolution · Vimeo-based dataset ×4 | PSNR: 25.27dB -> 25.34dB | Helped |
| add BSSR (ablation) | CSCSR · REDS-based dataset ×2 | PSNR: 33.04 -> 33.39 | Helped |
| add global perception to BSSR (ablation) | CSCSR · REDS-based dataset ×2 | PSNR: 33.17 -> 33.39 | Helped |
| use BSSR instead of interval-based representation (ablation) | CSCSR · REDS-based dataset ×2 | PSNR: 32.05 -> 33.39 | Helped |
| use joint rather than independent motion estimation (ablation) | CSCSR · REDS-based dataset ×2 | PSNR: 33.22 -> 33.39 | Helped |
| use joint motion estimation rather than assembled motion estimation (ablation) | CSCSR · REDS-based dataset ×2 | PSNR: 30.24 -> 33.39 | Helped |
Surprising findings
- Adapting gray-scale spike SR methods per color channel (VidarSR*, SpikeSR-Net*) performs worse than their original versions, highlighting the importance of color correlation.
Limitations
- Evaluation relies on simulated spike streams, with only a limited set of real-world captures for qualitative checks.
- Motion estimation errors still affect temporal pixel search, motivating the refinement module.
Experimental setup
Datasets: REDS Vid4 Vimeo
Benchmarks: REDS-based dataset Vid4-based dataset Vimeo-based dataset real-world captured spike streams
Metrics: PSNR SSIM LPIPS
Baselines: TFI+TSCNN TFP+TSCNN TFI+Real-RawVSR TFP+Real-RawVSR 3DRI+SwinIR 3DRI+BasicVSR VidarSR SpikeSR-Net
Builds on: Spike2Flow TFI TFP SwinIR BasicVSR
Closest papers by meaning
- Self-Supervised Learning for Color Spike Camera Reconstruction (CVPR 2025 · similarity 0.92)
- Joint Demosaicing and Denoising for Spike Camera (AAAI 2024 · similarity 0.90)
- Spk2SRImgNet: Super-Resolve Dynamic Scene from Spike Stream via Motion Aligned Collaborative Filtering (CVPR 2025 · similarity 0.87)
- Rethinking High-speed Image Reconstruction Framework with Spike Camera (AAAI 2025 · similarity 0.86)
- Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations (CVPR 2024 · similarity 0.84)
- Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.