Research map / Image generation / Super-resolution
Spk2SRImgNet: Super-Resolve Dynamic Scene from Spike Stream via Motion Aligned Collaborative Filtering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spk2SRImgNet, an end-to-end network that super-resolves spike-camera streams into HR images using a motion aligned collaborative filtering module to suppress spike fluctuations.
Problem
Spike camera super resolution is hampered by low spatial resolution and by fluctuations in spike intervals caused by Poisson shot noise, readout quantization and thermal noise, which make features noisy and temporally incoherent.
Approach
Adapts classic non-local collaborative filtering (BM3D-style) to spike streams: features from neighboring moments are motion-aligned with deformable convolution, stacked and filtered jointly in a learned transform domain, then inverse-warped back and aggregated with the initial features.
Key results
- On REDS-based x4 SR, Spk2SRImgNet reaches 29.50 dB PSNR / 0.8369 SSIM, above SpikeSR-Net (29.22 dB) and VidarSR (28.30 dB), at 3.86M params and 0.21 s runtime.
- Ablation on REDS x4 shows MACF adds 0.39 dB over the SIR-only baseline (29.07 to 29.46 dB without the final step-5 variant comparison, 29.50 dB for the full model).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Spk2SRImgNet instead of SpikeSR-Net | SCSR · REDS-based ×2 | PSNR: 34.29 -> 34.50 | Helped |
| Use Spk2SRImgNet instead of SpikeSR-Net | SCSR · REDS-based ×4 | PSNR: 29.22 -> 29.50 | Helped |
| Add the SIR module (ablation) | Spk2SRImgNet · REDS-based ×4 | PSNR: 29.46 -> 29.50 | Helped |
| Add MACF (ablation) | Spk2SRImgNet without SIR or MACF · REDS-based ×4 | PSNR: 29.07 -> 29.46 | Helped |
| Add MACF (ablation) | Spk2SRImgNet with SIR · REDS-based ×4 | PSNR: 29.15 -> 29.50 | Helped |
| Use MACF step 3 alone (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.07 -> 29.17 | Helped |
| Use MACF step 4 alone (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.07 -> 29.32 | Helped |
| Use MACF steps 3 and 4 (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.07 -> 29.34 | Helped |
| Use MACF steps 3 and 5 (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.07 -> 29.35 | Helped |
| Use MACF steps 4 and 5 (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.07 -> 29.33 | Helped |
| Add MACF step 5 to steps 3 and 4 (ablation) | Spk2SRImgNet ablation · REDS-based ×4 | PSNR: 29.34 -> 29.46 | Helped |
Limitations
- Training and evaluation rely on synthesized spike streams from a simulator; real-world evaluation is qualitative only without ground truth.
- Similarity exploited is only temporal; spatial non-local similarity is left for future work.
Experimental setup
Datasets: REDS Adobe240
Benchmarks: REDS-based SCSR evaluation dataset Adobe240-based SCSR evaluation dataset
Metrics: PSNR SSIM LPIPS runtime
Baselines: VidarSR SpikeSR-Net TFP+DAT Spk2ImgNet+DAT WGSE+DAT BSF+DAT TFP+BasicVSR++ Spk2ImgNet+BasicVSR++ WGSE+BasicVSR++ BSF+BasicVSR++
Builds on: BM3D BasicVSR++ Spike interval based reconstruction (SIR) DAT Deformable convolution
Closest papers by meaning
- Spatio-Temporal Interactive Learning for Efficient Image Reconstruction of Spiking Cameras (NeurIPS 2024 · similarity 0.90)
- SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams (NeurIPS 2024 · similarity 0.88)
- Spike-guided Motion Deblurring with Unknown Modal Spatiotemporal Alignment (CVPR 2024 · similarity 0.88)
- Rethinking High-speed Image Reconstruction Framework with Spike Camera (AAAI 2025 · similarity 0.87)
- Super-Resolution Reconstruction from Bayer-Pattern Spike Streams (CVPR 2024 · similarity 0.87)
- Joint Demosaicing and Denoising for Spike Camera (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.