Research map / Detection and segmentation / CNN features
CREST: An Efficient Conjointly-trained Spike-driven Framework for Event-based Object Detection Exploiting Spatiotemporal Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CREST, a spike-driven framework combining a conjoint learning rule (surrogate DL-Net for backprop), a multi-scale spatiotemporal event integrator (MESTOR), and an ST-IoU loss on few-spikes neurons for energy-efficient event-based object detection and recognition.
Problem
Deep SNNs trained for event-based object recognition and detection suffer from gradient vanishing and high spatiotemporal BP cost, and existing event representations and IoU losses handle multi-scale spatiotemporal event features poorly, causing redundancy and reduced accuracy.
Approach
A conjoint learning rule trains FSN-based SNNs by backpropagating through a surrogate DL-Net with discrete-level activations (spatial-only gradient, identity for round), while inference stays spike-driven; MESTOR extracts spatiotemporally continuous events at multiple scales, and ST-IoU adds a spike-density term to CIoU.
Key results
- On Gen1, CREST (CSPdarknet-tiny) reaches mAP50:95 0.360 and mAP50 0.632 with 0.04 mJ... specifically 6.31 mJ energy versus SFOD 0.593 mAP50 at 124.73 mJ, roughly 20x lower energy while improving mAP50.
- On NCARs, CREST (DenseNet121-16) reaches 0.952 accuracy with 1.95M parameters and 0.04 mJ energy, outperforming the best prior SNN (DenseNet121-16 VoxelCube, 0.937).
- On PKU-Vidar-DVS, CREST variants reach mAP50 0.425-0.432, surpassing the ANN baselines (up to 0.386).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace FSN-BP with CLR (ablation) | CREST · NCARs | accuracy: 0.948 -> 0.952 | Helped |
| replace FSN-BP with CLR (ablation) | Gen1 baseline · Gen1 | mAP50:95: 0.269 -> 0.279 | Helped |
| add MESTOR to the CLR baseline (ablation) | CREST-A · Gen1 | mAP50:95: 0.279 -> 0.358 | Helped |
| add ST-IoU (ablation) | CREST-B · Gen1 | mAP50:95: 0.358 -> 0.360 | Helped |
| combine spatial-temporal continuity component A with spatial and temporal components B (ablation) | MESTOR · NCARs | accuracy: 0.873 -> 0.949 | Helped |
| add component B to component A (ablation) | MESTOR · NCARs | firing rate: 0.164 -> 0.165 | Mixed |
| add spatial and temporal components B to component A (ablation) | MESTOR · NCARs | accuracy: 0.873 -> 0.949 | Helped |
| replace FSN-ReLU with FSN-LReLU (ablation) | CREST · NCARs | accuracy: 0.949 -> 0.949 | No clear effect |
| replace FSN-ReLU with FSN-LReLU (ablation) | CREST · NCARs | firing rate: 0.165 -> 0.310 | Hurt |
| use ST-IoU instead of CIoU (ablation) | CREST · Gen1 | mAP | Helped |
| include the Spiking-IoU component in ST-IoU (ablation) | CREST · Gen1 | mAP | Helped |
| include the CIoU component in ST-IoU (ablation) | CREST · Gen1 | convergence speed | Helped |
Surprising findings
- FSN-LReLU (leaky ReLU emulation) yields similar accuracy to FSN-ReLU but substantially higher firing rates, so ReLU was chosen.
Limitations
- Energy results come from an analytical energy model rather than measured hardware energy.
- Largest mAP on Gen1 (0.504 for EGO-12) is achieved by a much larger ANN, which CREST does not match.
Experimental setup
Datasets: NCARs Gen1 PKU-Vidar-DVS
Benchmarks: NCARs Gen1 PKU-Vidar-DVS
Metrics: accuracy mAP50:95 mAP50 firing rate energy (mJ)
Baselines: SFOD EMS-SNN VoxelCube SNN (VC-Dense/VC-Mobile) Asynet S-Center LT-SNN TR-YOLO TAR-events
Builds on: few-spikes neuron YOLOv4 CIoU STBP
Closest papers by meaning
- Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention (CVPR 2025 · similarity 0.91)
- SFOD: Spiking Fusion Object Detector (CVPR 2024 · similarity 0.91)
- Hybrid Spiking Vision Transformer for Object Detection with Event Cameras (ICML 2025 · similarity 0.90)
- SpikingYOLOX: Improved YOLOX Object Detection with Fast Fourier Convolution and Spiking Neural Networks (AAAI 2025 · similarity 0.89)
- SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition (ICLR 2024 · similarity 0.89)
- Brain-Inspired Spiking Neural Networks for Energy-Efficient Object Detection (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.