Research map / 3D vision / Depth and stereo
Transient Glimpses: Unveiling Occluded Backgrounds through the Spike Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SpkOccNet, which removes foreground occlusions from a single fast-moving spike camera's spike stream using long-short-window feature extraction and cross-view mutual attention, and introduces the real-world S-OCC dataset.
Problem
Recovering clear backgrounds behind dense foreground occlusions (de-occlusion / synthetic aperture imaging) when frame-based multi-view captures suffer from limited viewpoints and motion blur under fast motion.
Approach
Uses a single moving spike camera to capture a dense continuous stream of spikes as multi-view information, splits the stream into end and central segments processed with dense and long-window representations, and fuses them with a cross-view mutual window attention module.
Key results
- SpkOccNet reaches 26.83 dB PSNR and 0.805 SSIM on S-OCC, versus 25.51 dB for E-SAI and 24.77 dB for DeOccNet.
- SpkOccNet has about 4.9M parameters, roughly 8.0x fewer than DeOccNet (39.04M) and 3.8x fewer than E-SAI (18.59M).
- Using only 45 ms of spikes still outperforms image inputs (26.42 dB vs 25.90 dB PSNR).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SpkOccNet instead of E-SAI | de-occlusion on S-OCC · S-OCC Total | PSNR: 25.51 -> 26.83 | Helped |
| use SpkOccNet instead of E-SAI | de-occlusion on S-OCC · S-OCC Square Mesh | PSNR: 28.26 -> 27.35 | Hurt |
| add dense-window center representation to long-window and end-view features (ablation) | SpkOccNet · S-OCC Total | PSNR: 25.53 -> 26.11 | Helped |
| add long-window and end-view representations to dense center features (ablation) | SpkOccNet · S-OCC Total | PSNR: 23.72 -> 26.11 | Helped |
| add mutual-window attention (MMVA) for feature fusion (ablation) | SpkOccNet · S-OCC Total | PSNR: 26.11 -> 26.83 | Helped |
| increase spike input length from 45ms to 75ms (ablation) | SpkOccNet · S-OCC Total | PSNR: 26.42 -> 26.77 | Helped |
| increase spike input length from 75ms to 105ms (ablation) | SpkOccNet · S-OCC Total | PSNR: 26.77 -> 26.83 | Helped |
| replace spike input with simulated images (ablation) | SpkOccNet · S-OCC Total | PSNR: 26.83 -> 25.90 | Hurt |
Surprising findings
- Even 45 ms of spike input (one frame-camera exposure) outperforms simulated image inputs of the same model.
Limitations
- Evaluation limited to one spike camera setup with five occlusion types on a single dataset.
- Event-based SAI methods require camera intrinsics and depth priors, which the approach avoids but the comparison is only against three methods.
Experimental setup
Datasets: S-OCC
Benchmarks: S-OCC
Metrics: PSNR SSIM
Baselines: Baseline U-Net DeOccNet E-SAI
Builds on: Restormer Swin Transformer E-SAI DeOccNet
Closest papers by meaning
- SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams (NeurIPS 2024 · similarity 0.86)
- Rethinking High-speed Image Reconstruction Framework with Spike Camera (AAAI 2025 · similarity 0.86)
- Spike-guided Motion Deblurring with Unknown Modal Spatiotemporal Alignment (CVPR 2024 · similarity 0.86)
- Spk2SRImgNet: Super-Resolve Dynamic Scene from Spike Stream via Motion Aligned Collaborative Filtering (CVPR 2025 · similarity 0.85)
- Finding Visual Saliency in Continuous Spike Stream (AAAI 2024 · similarity 0.85)
- Spatio-Temporal Interactive Learning for Efficient Image Reconstruction of Spiking Cameras (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.