Research map / Video / Video-language understanding
Abductive Ego-View Accident Video Understanding for Safe Driving Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MM-AU, a large ego-view multimodal accident video dataset with object boxes, accident reason QA, and text descriptions, plus AdVersa-SD with an abductive CLIP and an object-centric accident video diffusion model (OAVD).
Problem
Ego-view traffic accident video understanding lacks large-scale datasets with aligned text (accident reasons, prevention advice, categories) and object annotations, and existing work does not support abductive reasoning about accident cause-effect chains for safe driving.
Approach
Pairs temporally segmented accident videos (normal, near-accident, accident) with text descriptions and trains an abductive CLIP with a contrastive interaction loss over virtual negative co-occurrence pairs, which drives an object-centric video diffusion model that uses bounding boxes and a masked latent reconstruction loss.
Key results
- SeViLA fine-tuned on MM-AU ArA reaches 89.02% test accuracy, surpassing CoVGT (79.97%) by about 7% to 9%.
- OAVD with the abductive CLIP reaches FVD 5238.1 and CLIP score 27.24, versus 9545.6 FVD and 21.77 CLIP score for Tune-A-Video.
- DiffusionDet and YOLOv5s are the leading object detectors; DiffusionDet shows superior performance on the accident-window test set under V2-Train.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Abductive-CLIP instead of Sequential-CLIP trained only on positive co-occurrence pairs (ablation) | OAVD · MM-AU | CLIP score: 27.14 -> 27.24 | Helped |
| use Abductive-CLIP instead of Sequential-CLIP trained only on positive co-occurrence pairs (ablation) | OAVD · MM-AU | FVD: 5372.3 -> 5238.1 | Helped |
| add object bounding-box input (ablation) | OAVD · MM-AU video diffusion | FVD | Helped |
| use SeViLA with Flan T5-XL instead of CoVGT | MM-AU accident reason answering · MM-AU test | accuracy: 79.97 -> 89.02 | Helped |
Surprising findings
- Most detectors degrade on accident windows, and DiffusionDet trained only on accident-free windows generalizes better to accident windows than other detectors.
- Fine-grained object-level video representations are key to higher ArA accuracy, but most methods fail on cyclist-related reasons.
Limitations
- Accident reason and prevention advice annotations cover only 110 sentence pairs, and the video diffusion evaluation uses a subset of 6000 training pairs and 1500 generated clips.
- Video diffusion quality is measured by FVD and CLIP score only, with no human evaluation.
- Real-synthetic gap is avoided, but many accident videos come from several source datasets with uneven category coverage (some categories have very few samples).
Experimental setup
Datasets: MM-AU BDD-100K CCD A3D DoTA DADA-2000 COCO
Benchmarks: MM-AU
Metrics: mAP50 AR accuracy FVD CLIP score FPS
Baselines: Tune-A-Video ControlVideo original CLIP Sequential-CLIP HCRN ClipBERT VGT FrozenGQA CoVGT SeViLA FasterRCNN CornerNet
Builds on: CLIP XCLIP Latent Diffusion Model Tune-A-Video DiffusionDet
Closest papers by meaning
- SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models (ICML 2025 · similarity 0.85)
- iFinder: Structured Zero-Shot Vision-Based LLM Grounding for Dash-Cam Video Reasoning (NeurIPS 2025 · similarity 0.84)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.83)
- TUMTraf VideoQA: Dataset and Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenes (ICML 2025 · similarity 0.83)
- Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Events (CVPR 2025 · similarity 0.83)
- H-MBA: Hierarchical MamBa Adaptation for Multi-Modal Video Understanding in Autonomous Driving (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.