Research map / 3D vision / Depth and stereo
Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BSF, a spike camera image reconstruction network that uses multi-order DSFT fusion to suppress spike fluctuations and multi-granularity (patch cross-attention + deformable conv) alignment, and releases a synthetic REDS-SCIR benchmark.
Problem
Spike camera image reconstruction is hurt by spike fluctuations caused by Poisson photon arrival, thermal noise, and clock quantization, which make spike intervals unstable and unreliable for estimating light intensity.
Approach
Shows that the reciprocal of the (1,1)-order differential of spike firing time is an unbiased estimator of light intensity under quantization, and extends DSFT to multiple orders fused in a reciprocal domain; aligns inter-moment features coarse-to-fine with patch-level cross-attention using local search followed by deformable convolution.
Key results
- On REDS-SCIR at light intensity 1.00, BSF reaches 39.76 PSNR and 0.970 SSIM, above the retrained Spk2ImgNet (39.16) and WGSE (38.97).
- On real-captured spike data, BSF achieves the best BRISQUE (18.529), PIQE (23.477) and HOSA (25.523) among compared methods.
- Ablation shows multi-order DSFT input improves PSNR from 38.44 (binary spikes) to 38.78 (D(1,1)) and up to 39.76 with full MODF plus MGA with 3 pyramid levels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace binary spike input with multi-order DSFT fusion (ablation) | BSF · REDS-SCIR | PSNR: 38.44 -> 39.76 | Helped |
| use multi-order DSFT instead of only D(1,1) (ablation) | BSF · REDS-SCIR | PSNR: 38.78 -> 39.76 | Helped |
| add MGA (ablation) | BSF with multi-order DSFT · REDS-SCIR | PSNR: 38.99 -> 39.76 | Helped |
| add CAPA to DCN alignment (ablation) | BSF MGA · REDS-SCIR | PSNR: 38.91 -> 39.06; 39.02 -> 39.39; 39.38 -> 39.76 | Helped |
| increase MGA pyramid levels from 1 to 3 (ablation) | BSF MGA · REDS-SCIR | PSNR: 39.06 -> 39.76 | Helped |
| increase input sub-streams from 21 to 61 (ablation) | BSF · REDS-SCIR | PSNR: 38.14 -> 39.76 | Helped |
| increase input frames from 61 to 81 (ablation) | BSF · REDS-SCIR | PSNR: 39.76 -> 39.70 | Hurt |
| use BSF instead of the strongest listed retrained baseline | REDS-SCIR, η = 1.00 · REDS-SCIR, η = 1.00 | PSNR: 39.16 -> 39.76 | Helped |
| use BSF instead of the strongest listed retrained baseline | REDS-SCIR, η = 0.75 · REDS-SCIR, η = 0.75 | PSNR: 38.27 -> 39.09 | Helped |
| use BSF instead of the strongest listed retrained baseline | REDS-SCIR, η = 0.50 · REDS-SCIR, η = 0.50 | PSNR: 36.75 -> 37.76 | Helped |
| use BSF instead of the strongest listed baseline | Real-captured data · Real-captured data | BRISQUE: 23.429 -> 18.529 | Helped |
| use BSF instead of the strongest listed baseline | Real-captured data · Real-captured data | PIQE: 25.076 -> 23.477 | Helped |
Surprising findings
- The training-free TFP baseline with a temporal mean reaches competitive PSNR at high light intensity compared with some learned methods, and the paper notes retraining baselines on REDS-SCIR changes their ranking considerably.
Limitations
- Evaluation on real data uses only no-reference metrics since no ground truth exists.
- Synthetic benchmark relies on a simulated Poisson and quantization model with fixed parameters.
- Relies on a window of 61 input frames, with performance converging only at that size.
Experimental setup
Datasets: REDS REDS-SCIR
Benchmarks: REDS-SCIR real-captured spike data (12 scenes)
Metrics: PSNR SSIM LPIPS BRISQUE PIQE HOSA
Baselines: TFP TFI TFSTP MAHTF FireNet ETNet SSML Spk2ImgNet WGSE SNM
Builds on: Spike2Flow DSFT Spk2ImgNet deformable convolution EMA-VFI
Closest papers by meaning
- Optical Flow for Spike Camera with Hierarchical Spatial-Temporal Spike Fusion (AAAI 2024 · similarity 0.90)
- Rethinking High-speed Image Reconstruction Framework with Spike Camera (AAAI 2025 · similarity 0.88)
- SpikeNeRF: Learning Neural Radiance Fields from Continuous Spike Stream (CVPR 2024 · similarity 0.87)
- USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting (CVPR 2025 · similarity 0.87)
- Intensity-Robust Autofocus for Spike Camera (CVPR 2024 · similarity 0.86)
- Recognizing Ultra-High-Speed Moving Objects with Bio-Inspired Spike Camera (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.