Research map / Detection and segmentation / CNN features
Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an attention-based hybrid SNN-ANN backbone (with an attention SNN-ANN bridge and optional DWConvLSTM multi-timescale variant) for event-based object detection, evaluated on Gen1/Gen4 and partly deployed on Loihi 2.
Problem
SNNs are efficient for event-camera data but lag ANNs in object detection accuracy, and there is no effective bridge converting sparse spike features into dense representations for ANN layers in benchmark detection tasks.
Approach
An attention-based SNN-ANN bridge (ASAB) combining Spatial-aware Temporal attention (time-wise separable deformable convolution plus temporal self-attention) and Event-rate Spatial attention converts spike tensors into dense feature maps for ANN blocks; an optional DWConvLSTM stage on the bridge output captures slower dynamics.
Key results
- Proposed hybrid model reaches mAP 0.35 on Gen1 with 6.6M parameters, outperforming SNN methods such as EMS-RES34 (0.31, 14.4M) and matching ANN Events-RetinaNet (0.34, 33M).
- On Gen4 the hybrid model achieves mAP 0.27, above E2Vid-RetinaNet (0.25) and AEGNN (0.16).
- Ablation: removing the bridge module ASAB drops mAP from 0.35 to 0.30; the SNN block on Intel Loihi 2 runs at about 1.7 W and 1.9 ms per step.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the hybrid SNN-ANN method instead of the strongest listed SNN baseline | Gen1 object detection · Gen1 Automotive Detection | mAP: 0.31 -> 0.35 | Helped |
| use the hybrid SNN-ANN method instead of Events-RetinaNet | Gen1 object detection · Gen1 Automotive Detection | mAP: 0.34 -> 0.35 | Helped |
| use the hybrid SNN-ANN method instead of Inception + SSD or RRC-Events | Gen4 object detection · Gen4 Automotive Detection | mAP: 0.34 -> 0.27 | Hurt |
| add DWConvLSTM units to the hybrid backbone | Proposed hybrid model · Gen1 Automotive Detection | mAP: 0.35 -> 0.43 | Mixed |
| add temporal attention Φta (ablation) | ASAB · Gen1 Automotive Detection | mAP: 0.33 -> 0.35 | Helped |
| use deformable convolution instead of standard convolution (ablation) | ASAB · Gen1 Automotive Detection | mAP: 0.34 -> 0.35 | Helped |
| add Event-Rate Spatial attention (ablation) | ASAB · Gen1 Automotive Detection | mAP: 0.34 -> 0.35 | Helped |
| use ASAB instead of simple accumulation (ablation) | hybrid backbone · Gen1 Automotive Detection | mAP: 0.30 -> 0.35 | Helped |
| add the βasab bridge module (ablation) | Baselineann · Gen1 Automotive Detection | mAP(.5): 0.53 -> 0.61 | Helped |
| increase one SNN layer and reduce one ANN layer (ablation) | Proposed hybrid model · Gen1 Automotive Detection | mAP(.5): 0.61 -> 0.58 | Hurt |
| quantize convolution kernel weights from float16 to int8 (ablation) | Proposed model on Loihi 2 · Gen1 Automotive Detection | mAP: 0.348 -> 0.343 | Hurt |
Surprising findings
- The RNN variant (Proposed+RNN, 0.43 mAP) is below the RNN baselines such as RVT-B (0.47) on Gen1.
Limitations
- Comparison to other methods relies on numbers reported in prior papers because most lack open-source code.
- The RNN variant trails the strongest RNN-based baselines on Gen1.
- Loihi 2 benchmarking uses simulated inputs due to chip IO limitations.
Experimental setup
Datasets: Gen1 Automotive Detection dataset Gen4 Automotive Detection dataset
Benchmarks: Gen1 Automotive Detection Gen4 Automotive Detection
Metrics: mAP mAP(.5) parameter count MACs power (mJ)
Baselines: Inception+SSD RRC-Events Events-RetinaNet E2Vid-RetinaNet RVT-B AEGNN SparseConv EMS-RES34 SpikeFPN DenseNet121+SSD RED ASTMNet
Builds on: RVT YOLOX SpikingJelly PLIF neurons deformable convolution RED
Closest papers by meaning
- SFOD: Spiking Fusion Object Detector (CVPR 2024 · similarity 0.93)
- CREST: An Efficient Conjointly-trained Spike-driven Framework for Event-based Object Detection Exploiting Spatiotemporal Dynamics (AAAI 2025 · similarity 0.91)
- Hybrid Spiking Vision Transformer for Object Detection with Event Cameras (ICML 2025 · similarity 0.91)
- Brain-Inspired Spiking Neural Networks for Energy-Efficient Object Detection (CVPR 2025 · similarity 0.89)
- Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset (CVPR 2025 · similarity 0.89)
- SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.