Research map / Video / Video segmentation and tracking
Seeing Motion at Nighttime with an Event Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NER-Net, an event-to-image reconstruction network for nighttime dynamic scenes that models event temporal trailing and non-uniform illumination, trained on a new co-axial real low-light event dataset (RLED).
Problem
Reconstructing high-quality images from event cameras under nighttime low-light conditions, where events show temporal trailing and spatially non-uniform distributions due to artificial lighting, and paired real low-light event/image data is lacking.
Approach
Models nighttime event statistics via a learnable event timestamp calibration module (LETC) that suppresses trailing events, and a non-uniform illumination aware module (NIAM) with local adaptation gate, global context block and spatiotemporal aggregation unit; trained on a co-axial real paired low-light event dataset.
Key results
- On RLED, NER-Net reaches MSE 0.011, SSIM 0.717, LPIPS 0.309, outperforming E2VID+ and ET-Net trained on RLED (MSE 0.019, SSIM 0.688, LPIPS 0.393 for E2VID+ on RLED).
- On real nighttime dynamic scenes, NER-Net achieves LOE 92.81 versus 99.78 for SCI and 113.02 for KinD++, with a user study score of 6.02 vs 5.03 for the next best.
- Ablation: LETC and NIAM each improve RLED SSIM; full model reaches SSIM 0.717 and LPIPS 0.309 from a baseline of 0.688 and 0.393.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use NER-Net instead of the strongest listed baseline | event-based image reconstruction · RLED | MSE: 0.019 -> 0.011 | Helped |
| use NER-Net instead of the strongest listed baseline | event-based image reconstruction · DSEC-night | LOE: 1165.1 -> 1031.2 | Helped |
| use NER-Net instead of the strongest listed baseline | event-based image reconstruction · MVSEC-night | LPIPS: 0.534 -> 0.502 | Helped |
| use NER-Net instead of the strongest listed baseline | event-based image reconstruction · VECtor-hdr | SSIM: 0.317 -> 0.344 | Helped |
| replace raw events with ETS-processed events without LETC or NIAM (ablation) | NER-Net · RLED | MSE: 0.019 -> 0.045 | Hurt |
| add LETC (ablation) | NER-Net without ETS or NIAM · RLED | MSE: 0.019 -> 0.017 | Helped |
| add ETS alongside LETC (ablation) | NER-Net without NIAM · RLED | MSE: 0.017 -> 0.016 | Helped |
| add LAG (ablation) | NER-Net with ETS and LETC · RLED | MSE: 0.016 -> 0.014 | Helped |
| add GCB (ablation) | NER-Net with ETS, LETC, and LAG · RLED | MSE: 0.014 -> 0.015 | Hurt |
| add SAU (ablation) | NER-Net with ETS, LETC, LAG, and GCB · RLED | MSE: 0.015 -> 0.011 | Helped |
Surprising findings
- Models trained on simulated events (V2E) generalize poorly to real nighttime events, underscoring the sim-to-real gap caused by non-uniform lighting.
- Directly reconstructing from ETS-processed events performs worse than the learned LETC, indicating hand-designed trail suppression alone is insufficient.
Limitations
- Fails in extreme low-light scenes below about 0.5 lux where events are too sparse.
- Does not model color information.
- Dataset collected under daytime lighting with an ND filter to simulate low light rather than true nighttime captures.
Experimental setup
Datasets: RLED DSEC MVSEC VECtor V2E simulated data ESIM
Benchmarks: RLED DSEC-night MVSEC-night VECtor-hdr
Metrics: MSE SSIM LPIPS LOE NIQE SPAQ user study score
Baselines: E2VID+ ET-Net DVS-Dark KinD++ SCI URetinex-Net
Builds on: E2VID+ ConvLSTM PredRNN Event Spike Tensor V2E
Closest papers by meaning
- Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach (CVPR 2024 · similarity 0.88)
- Event Fields: Capturing Light Fields at High Speed, Resolution, and Dynamic Range (CVPR 2025 · similarity 0.87)
- LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising (CVPR 2024 · similarity 0.86)
- Event-3DGS: Event-based 3D Reconstruction Using 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- Latency Correction for Event-guided Deblurring and Frame Interpolation (CVPR 2024 · similarity 0.86)
- EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.