Research map / Detection and segmentation / CNN features
Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MvHeat-DET, a Mixture-of-Experts heat-conduction backbone (DFT/DCT/HT experts with Gumbel-softmax routing) with IoU-based query selection for event-camera object detection, and releases the EvDET200K benchmark with 15+ re-trained detectors.
Problem
Object detection from event camera streams lacks large high-resolution benchmarks, and existing event detectors (CNN, Transformer, SNN) face limited receptive fields, high computational cost, or weaker accuracy.
Approach
Replace the attention blocks of a DETR-style detector with a mixture-of-experts heat conduction operator that routes features through DFT, DCT, or Haar transform experts, with learnable thermal diffusivity predicted from frequency embeddings, plus an IoU-aware query selection module.
Key results
- MvHeat-DET reaches 52.9 mAP@50:95 on EvDET200K, versus 50.3 for the vHeat-encoder baseline and 49.1 for Swin-T, with 47.5M params and 56.4G FLOPs.
- On N-Caltech101, MvHeat-DET reports 55.7 mAP, above Jeziorek et al. (53.4) and EAS-SNN (53.8).
- Ablation: adding the MoE experts raises mAP from 50.3 (DCT only) to 52.9 (DCT+DFT+HT); learnable thermal diffusivity predicted from frequency embeddings reaches 49.7 versus 48.2 with fixed k.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IoU-based query selection (ablation) | DETR on EvDET200K · EvDET200K | mAP: 40.9 -> 41.6 | Helped |
| replace the Transformer encoder with vHeat (ablation) | DETR + IQS on EvDET200K · EvDET200K | mAP: 41.6 -> 50.3 | Helped |
| add the MoE strategy to the encoder (ablation) | DETR + IQS + vHeat encoder on EvDET200K · EvDET200K | mAP: 50.3 -> 52.9 | Helped |
| vary the number of MHCO modules in stage three from 6 to 12, 18, and 24 (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 52.6; 52.9; 52.9; 53.4 | Helped |
| increase the expert count from one DCT expert to DCT+DFT and then DCT+DFT+HT (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 50.3 -> 52.7 -> 52.9 | Helped |
| use DFT alone instead of the full expert combination (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 52.9 -> 48.8 | Hurt |
| use DCT alone instead of the full expert combination (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 52.9 -> 50.3 | Hurt |
| use HT alone instead of the full expert combination (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 52.9 -> 50.6 | Hurt |
| combine DCT and DFT experts (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 50.3 -> 52.7 | Helped |
| combine DCT and HT experts (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 50.3 -> 52.4 | Helped |
| combine DFT and HT experts (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 48.8 -> 51.9 | Helped |
| combine all three experts, DCT, DFT, and HT (ablation) | MvHeat-DET on EvDET200K · EvDET200K | mAP: 52.9 | Helped |
Surprising findings
- The SNN-based and YOLO baselines trail several Transformer and CNN detectors on EvDET200K, with EMS-YOLO at 32.1 mAP.
- Swin-T (49.0 mAP) and DetectoRS (49.1) are close to the proposed model's 52.9 mAP, with the gap mostly coming from the heat-conduction backbone.
Limitations
- Only three transform experts (DFT, DCT, HT) are considered, and the policy network's routing behavior is not analyzed in the main text.
- Evaluation on N-Caltech101 is compared against a limited set of prior methods with different formats (event points vs. frames).
- Dataset scale is 10 classes from one camera model (Prophesee EVK4-HD), limiting generality.
Experimental setup
Datasets: EvDET200K N-Caltech101 Gen1 1Mpx
Benchmarks: EvDET200K N-Caltech101
Metrics: mAP@50:95 mAP@50 mAP@75 precision recall params FLOPs FPS
Baselines: Faster R-CNN S5-ViT SAST SpikeYOLO YOLOv10 RVT EMS-YOLO YOLOv6 Swin-T DetectoRS RED DETR
Builds on: vHeat DETR Deformable DETR DINO
Closest papers by meaning
- Hybrid Spiking Vision Transformer for Object Detection with Event Cameras (ICML 2025 · similarity 0.89)
- SFOD: Spiking Fusion Object Detector (CVPR 2024 · similarity 0.89)
- Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention (CVPR 2025 · similarity 0.89)
- CREST: An Efficient Conjointly-trained Spike-driven Framework for Event-based Object Detection Exploiting Spatiotemporal Dynamics (AAAI 2025 · similarity 0.87)
- FlexEvent: Towards Flexible Event-Frame Object Detection at Varying Operational Frequencies (NeurIPS 2025 · similarity 0.87)
- LEOD: Label-Efficient Object Detection for Event Cameras (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.