Research map / Video / Video segmentation and tracking
Neuromorphic Event Signal-Driven Network for Video De-raining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an event-driven deep unfolding video de-raining network that uses event-camera streams as dynamic priors, with spiking mutual-enhancement modules, outperforming prior SOTA on SynHeavy25, SynLight25 and NTURain.
Problem
Video de-raining with CMOS intensity frames suffers from limited temporal resolution, which hampers capturing dynamic rainfall and moving objects, causing under- or over-de-raining.
Approach
Treat event-camera streams as dynamic priors inside a modified video de-raining objective (background, rain-streak, and moving-object layers), solve it with an HQS/ADMM-based optimization unfolded into a CNN backbone, and encode event streams with spiking (LIF-based) mutual enhancement modules.
Key results
- Achieves 1.24 dB PSNR improvement on SynHeavy25 over prior SOTA RMFD (33.94 vs 32.70) with about 39% of parameters (11M vs 29M).
- Improves SynLight25 by 1.18 dB (38.17 vs 36.99) and NTURain by 0.52 dB (39.44 vs 38.92).
- Ablation: adding event streams raises the RFMD-style baselines' PSNR by about 0.3 dB; removing events from the proposed model drops SynHeavy25 PSNR by 2.09 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add event streams (ablation) | Ours · SynHeavy25 | PSNR: 31.85 -> 33.94 | Helped |
| add event streams (ablation) | Ours · SynHeavy25 | SSIM: 0.9306 -> 0.9416 | Helped |
| add event streams (ablation) | DetailNet · SynHeavy25 | PSNR: 16.50 -> 17.32 | Helped |
| add event streams (ablation) | DetailNet · SynHeavy25 | SSIM: 0.5441 -> 0.5682 | Helped |
| add event streams (ablation) | JORDER · SynHeavy25 | PSNR: 23.28 -> 23.82 | Helped |
| add event streams (ablation) | JORDER · SynHeavy25 | SSIM: 0.7513 -> 0.7690 | Helped |
| add event streams (ablation) | DDCDNet · SynHeavy25 | PSNR: 28.47 -> 28.76 | Helped |
| add event streams (ablation) | DDCDNet · SynHeavy25 | SSIM: 0.8866 -> 0.8923 | Helped |
| add event streams (ablation) | TMICS · SynHeavy25 | PSNR: 28.90 -> 29.37 | Helped |
| add event streams (ablation) | TMICS · SynHeavy25 | SSIM: 0.8743 -> 0.8795 | Helped |
| add event streams (ablation) | RFMD · SynHeavy25 | PSNR: 32.70 -> 32.98 | Helped |
| add event streams (ablation) | RFMD · SynHeavy25 | SSIM: 0.9350 -> 0.9374 | Helped |
Surprising findings
- Removing event streams from the proposed model lowers PSNR by 2.09 dB while SSIM drops by 0.110, showing event information mainly affects PSNR.
Limitations
- Event streams lack high-frequency information, so gains depend on event data quality and simulation.
- Real-world evaluation is only qualitative on self-captured sequences.
Experimental setup
Datasets: SynHeavy25 SynLight25 NTURain
Benchmarks: SynHeavy25 SynLight25 NTURain
Metrics: PSNR SSIM
Baselines: DetailNet J4RNet SpacCNN JORDER DCSFN DGCNet DDCDNet TMICS RMFD ESTINet
Builds on: K-SVD half-quadratic splitting ADMM LIF neuron
Closest papers by meaning
- Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing (AAAI 2025 · similarity 0.85)
- Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining (CVPR 2024 · similarity 0.85)
- Boosting Image De-Raining via Central-Surrounding Synergistic Convolution (AAAI 2025 · similarity 0.84)
- Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention (CVPR 2025 · similarity 0.84)
- DeblurSR: Event-Based Motion Deblurring under the Spiking Representation (AAAI 2024 · similarity 0.84)
- Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.