Research map / Detection and segmentation / 3D detection and driving
Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Ev-3DOD, the first event-camera-based 3D object detector that fuses LiDAR, images and events to detect objects during sensor blind time, plus two 100 FPS event 3D detection datasets (Ev-Waymo, DSEC-3DOD).
Problem
LiDAR and camera sensors run at fixed low frame rates (10-20 Hz), leaving blind time between active timestamps during which 3D object detection cannot be performed and objects move substantially.
Approach
Virtual 3D Event Fusion projects event features onto LiDAR voxel centroids to create virtual 3D event features, fuses them with voxel features via ROI pooling to estimate an implicit motion field and 3D box motion during blind time, plus a motion confidence estimator that rescales box scores.
Key results
- On Ev-Waymo at 100 FPS online evaluation, Ev-3DOD reaches 48.06 mAP and 45.60 mAPH, far above online LiDAR/camera baselines (best online baseline around 33 mAP).
- On DSEC-3DOD, Ev-3DOD achieves 31.17 mAP and 26.54 mAPH, versus at most 13.94 mAP for online baselines.
- Ablation on Ev-Waymo: V3D-EF adds about +13.25 mAP and the motion confidence estimator adds +1.51 mAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add V3D-EF to the base model (ablation) | Ev-3DOD · Ev-Waymo, blind time, L2 | mAP: +13.25% | Helped |
| add V3D-EF to the base model (ablation) | Ev-3DOD · Ev-Waymo, blind time, L2 | mAPH: +12.54% | Helped |
| add the motion confidence estimator (ablation) | Ev-3DOD with V3D-EF · Ev-Waymo, blind time, L2 | mAP: +1.51% | Helped |
| add the motion confidence estimator (ablation) | Ev-3DOD with V3D-EF · Ev-Waymo, blind time, L2 | mAPH: +1.38% | Helped |
| add the non-empty mask in V3D-EF (ablation) | Ev-3DOD · Ev-Waymo, blind time, L2 | mAP: 42.57 -> 46.55 | Helped |
| add the non-empty mask in V3D-EF (ablation) | Ev-3DOD · Ev-Waymo, blind time, L2 | mAPH: 40.50 -> 44.22 | Helped |
| add LiDAR features (ablation) | V3D-EF base model · Ev-Waymo, blind time, L2 | mAP: 33.30 -> 37.04 | Helped |
| add event features (ablation) | V3D-EF base model · Ev-Waymo, blind time, L2 | mAP: 33.30 -> 42.23 | Helped |
| fuse LiDAR and event features after pooling rather than before pooling (ablation) | V3D-EF · Ev-Waymo, blind time, L2 | mAP: 43.95 -> 46.55 | Helped |
| test the 100 FPS-trained model at 200 FPS without additional training (ablation) | Ev-3DOD · Ev-Waymo, 200 FPS test | mAP: 48.06 -> 47.87 | No clear effect |
| test the 100 FPS-trained model at 300 FPS without additional training (ablation) | Ev-3DOD · Ev-Waymo, 300 FPS test | mAP: 48.06 -> 47.81 | No clear effect |
| use event data to predict blind-time motion | Ev-3DOD versus online fixed-frame-rate baselines · Ev-Waymo, blind time, L2 | mAP: 33.27 -> 48.06 | Helped |
Surprising findings
- Non-empty mask in V3D-EF matters: projecting events onto all voxels drops mAP from 46.55 to 42.57.
- Fusing LiDAR and events before ROI pooling underperforms fusing after pooling (43.95 vs 46.55 mAP).
Limitations
- No recurrent processing of past event information, so discontinuities can occur when fixed-rate sensors update at active timestamps.
- Ev-Waymo uses simulated events from Vid2E, and 100 FPS labels for blind time are interpolated from 10 FPS annotations.
- DSEC-3DOD evaluates only vehicle and pedestrian classes due to lack of cyclist data.
Experimental setup
Datasets: Waymo Open Dataset DSEC Ev-Waymo DSEC-3DOD
Benchmarks: Ev-Waymo DSEC-3DOD
Metrics: mAP mAPH AP APH
Baselines: VoxelNeXt HEDNet Focals Conv LoGoNet NeuralPCI + EMA NeuralPCI + CBM
Builds on: LoGoNet VoxelNeXt NeuralPCI voxel point centroid
Closest papers by meaning
- AsyncBEV: Cross-modal flow alignment in Asynchronous 3D Object Detection (ICLR 2026 · similarity 0.87)
- Learning to Detect Objects with a 1 Megapixel Event Camera (NeurIPS 2020 · similarity 0.87)
- Time3D: End-to-End Joint Monocular 3D Object Detection and Tracking for Autonomous Driving (CVPR 2022 · similarity 0.87)
- PIXOR: Real-Time 3D Object Detection From Point Clouds (CVPR 2018 · similarity 0.86)
- DOPS: Learning to Detect 3D Objects and Predict Their 3D Shapes (CVPR 2020 · similarity 0.86)
- DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.