Research map / Detection and segmentation / CNN features
When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multimodal asynchronous hybrid network fusing event-camera streams (asynchronous GNN, DAGr) with RGB ResNet features and a GRU-attention head to detect traffic anomalies with millisecond response times on ROL and DoTA.
Problem
Existing ego-view traffic anomaly detection methods optimize accuracy but neglect response time (inference latency plus detection delay), which is critical for autonomous driving safety.
Approach
Combine asynchronous event-graph processing (DAGr with spline convolutions) on event camera streams with RGB CNN features, fused at object level and temporally modeled by GRUs with attention, so that anomalies are flagged with low latency; also introduces mResponse, a multi-threshold detection delay metric.
Key results
- On ROL, the method reaches AUC 0.879 and AUC-Frame 0.736 versus 0.871 and 0.733 for TTHF, while running at 579 FPS versus TTHF's 3 FPS.
- mResponse drops to 1.17 s on ROL and 1.21 s on DoTA, compared with 2.35 s and 1.81 s for the best prior methods in the reported tables (e.g. FOL-Ensemble, MAMTCF).
- On the Rush-out subset, mTTA improves by 0.23 s over the state of the art, and mResponse improves by 0.19 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the GRU temporal module (ablation) | RGB + Event baseline · ROL | AUC: 0.805 to 0.817 | Helped |
| add the GRU temporal module (ablation) | RGB + Event baseline · ROL | AP: 0.479 to 0.508 | Helped |
| add the GRU temporal module (ablation) | RGB + Event baseline · ROL | mTTA: 1.44 to 1.98 seconds | Helped |
| add the attention module (ablation) | ROL anomaly detector with BBox and Object features · ROL | AUC: 0.839 -> 0.879 | Helped |
| add RGB features (ablation) | two-stage model with BBox, asynchronous events, GRU, and attention · ROL | AUC: 0.862 -> 0.868 | Helped |
| add asynchronous event streams instead of synchronous event streams (ablation) | two-stage model with BBox and GRU · ROL | AUC: 0.840 -> 0.851 | Helped |
| add attention (ablation) | two-stage model with BBox and asynchronous event streams · ROL | AUC: 0.851 -> 0.862 | Helped |
| replace optical flow with synchronous event streams (ablation) | two-stage model with BBox and GRU · ROL | AUC: 0.855 -> 0.840 | Hurt |
| use the two-stage architecture instead of the single-stage architecture (ablation) | Ours · ROL | AUC: 0.879 -> 0.868 | Hurt |
| increase network depth from 4 to 6 layers (ablation) | Ours · ROL | AUC: 0.879 -> 0.892 | Helped |
| increase network depth from 4 to 6 layers (ablation) | Ours · ROL | mResponse: 1.17 -> 1.56 | Hurt |
| replace CNN backbone with Swin Transformer (ablation) | Ours · ROL | AUC: 0.879 -> 0.881 | Helped |
Surprising findings
- TTHF achieves higher AUC-Frame on DoTA (0.847) than the proposed method (0.823), and the paper concedes a trade-off.
- Synchronous event frames gave the lowest AUC and mTTA in the two-stage ablation, while asynchronous GNN processing performed best.
Limitations
- Performance depends heavily on the object detector; blurry or small objects cause detection failures that propagate to anomaly scores.
- Event data for ROL and DoTA is synthetic, generated by v2e from RGB video; no real event anomaly dataset is used.
- Transformer backbones improve accuracy but reduce FPS and increase response time, showing an accuracy-latency trade-off.
Experimental setup
Datasets: ROL DoTA DSEC Rush-out
Benchmarks: ROL DoTA Rush-out
Metrics: AUC AP AUC-Frame mTTA mResponse FPS mAP
Baselines: ConvAE ConvLSTMAE AnoPred FOL MAMTCF AM-Net STFE TTHF MOVAD AED-MAE EfficientAD
Builds on: DAGr DoTA YOLOX v2e
Closest papers by meaning
- SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving (AAAI 2024 · similarity 0.85)
- FT-AED: Benchmark Dataset for Early Freeway Traffic Anomalous Event Detection (NeurIPS 2024 · similarity 0.85)
- Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving (CVPR 2025 · similarity 0.84)
- Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras (CVPR 2025 · similarity 0.84)
- Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes (AAAI 2024 · similarity 0.84)
- VETime: Vision Enhanced Zero-Shot Time Series Anomaly Detection (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.