Research map / Image generation / Image restoration
SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S-SDM, the first self-supervised spike-guided motion deblurring framework that turns a blurry RGB frame plus low-resolution spike stream into a sharp sequence, distilling a cascaded BSN+EDSR+physics pipeline into a lightweight deblur network trained with LPIPS and reblur losses.
Problem
Recovering sharp image sequences from a single blurry RGB image is ill-posed; supervised spike-guided deblurring methods trained on synthetic data degrade sharply on real-world captures due to domain gaps in spike density, spike generation, and blur formation.
Approach
Derives a physical relation between blurry RGB, short-exposure spike counts and the latent sharp frame (SDM), then builds a self-supervised cascade (blind-spot denoising, EDSR super-resolution, SDM) as teacher and distills it into a lightweight LDN with an LPIPS teacher loss plus a reblur consistency loss, so no ground-truth sharp frames are needed.
Key results
- On GOPRO at V_th=2 and V_th=4, S-SDM reaches 26.367 and 25.433 PSNR versus 14.406 and 11.621 for the supervised SpkDeblurNet, though SpkDeblurNet is best at V_th=1 (28.307 vs 26.893).
- The 0.23M-parameter LDN outperforms the SAN student (27.928 vs 27.283 PSNR) with fewer parameters (0.234M vs 2.36M) and fewer FLOPs (33.60G vs 107.84G).
- On the real RSB dataset, S-SDM achieves an average LIQE score of 2.1987 versus 1.8648 for SpkDeblurNet on single-frame restoration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use self-supervised S-SDM instead of SpkDeblurNet | GOPRO, V_th=1 · GOPRO, V_th=1 | PSNR: 28.307 -> 26.893 | Hurt |
| Use self-supervised S-SDM instead of RED+DASR | GOPRO · GOPRO, V_th=2 and V_th=4 | PSNR: 23.178 -> 26.367; 21.942 -> 25.433 | Helped |
| Fine-tune S-SDM on real-world data rather than use the supervised baseline | RSB · RSB Average | LIQE: 1.8648 -> 2.1987 | Helped |
| Replace SAN with LDN (ablation) | self-supervised spike-guided deblurring · GOPRO | PSNR: 27.283 -> 27.928 | Helped |
| Add BSN to SDM (ablation) | S-SDM · GOPRO | PSNR: 23.012 -> 24.634 | Helped |
| Add SR after BSN (ablation) | S-SDM · GOPRO | PSNR: 24.634 -> 26.144 | Helped |
| Add LDN after BSN and SR (ablation) | S-SDM · GOPRO | PSNR: 26.144 -> 27.928 | Helped |
| Add teacher loss when training with reblur loss (ablation) | LDN · GOPRO | PSNR: 23.102 -> 27.345 | Helped |
| Add reblur loss when training with teacher loss (ablation) | LDN · GOPRO | PSNR: 26.563 -> 27.345 | Helped |
| Increase reblur-loss weight from λ=10 to λ=50 (ablation) | LDN with teacher and reblur losses · GOPRO | PSNR: 27.345 -> 27.742 | Helped |
| Increase reblur-loss weight from λ=50 to λ=100 (ablation) | LDN with teacher and reblur losses · GOPRO | PSNR: 27.742 -> 27.928 | Helped |
| Increase reblur-loss weight from λ=100 to λ=200 (ablation) | LDN with teacher and reblur losses · GOPRO | PSNR: 27.928 -> 27.620 | Hurt |
Surprising findings
- The supervised SpkDeblurNet collapses to 14.406 PSNR at V_th=2 when trained only at V_th=1, showing severe sensitivity to spike density shift.
- Without the teacher loss, the LDN collapses toward an identity mapping of the blurry input.
Limitations
- Depends on strict spatial-temporal calibration between the RGB and spike cameras; misalignment causes color shifts and quality loss.
- Real-world evaluation relies on a non-reference metric (LIQE) since no ground-truth sharp sequences exist for RSB.
- Assumes a static scene over the short exposure and similar colors across adjacent pixels, which may break under large motion.
Experimental setup
Datasets: GOPRO RSB
Benchmarks: GOPRO RSB
Metrics: PSNR SSIM LIQE
Baselines: LEVS Motion-ETR BiT TRMD+DASR RED+DASR REFID+DASR SpkDeblurNet SAN (GEM)
Builds on: EDI Blind Spot Network EDSR LPIPS CBAM
Closest papers by meaning
- Spike-guided Motion Deblurring with Unknown Modal Spatiotemporal Alignment (CVPR 2024 · similarity 0.91)
- DeblurSR: Event-Based Motion Deblurring under the Spiking Representation (AAAI 2024 · similarity 0.90)
- Rethinking High-speed Image Reconstruction Framework with Spike Camera (AAAI 2025 · similarity 0.89)
- Self-Supervised Learning for Color Spike Camera Reconstruction (CVPR 2025 · similarity 0.88)
- Spk2SRImgNet: Super-Resolve Dynamic Scene from Spike Stream via Motion Aligned Collaborative Filtering (CVPR 2025 · similarity 0.88)
- SpikeNeRF: Learning Neural Radiance Fields from Continuous Spike Stream (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.