Research map / Video / Video-language understanding
Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Ex-VAD generates frame captions with BLIP-2, converts cleaned captions into video-level anomaly explanations with an LLM, fuses them with CLIP visual features, and aligns LLM-augmented label phrases for explainable fine-grained video anomaly detection.
Problem
Existing video anomaly detection methods are coarse-grained and lack explanations of anomalies; fine-grained category detection and interpretability remain limited.
Approach
Combine VLM-generated frame captions, cleaned via CLIP image-text alignment, and LLM-written video-level explanations as a text modality fused with CLIP visual features, plus LLM-expanded label phrases filtered by top-k similarity and aligned with video features for fine-grained classification.
Key results
- Fine-grained detection on UCF-Crime: Ex-VAD achieves AVG mAP 10.15 vs 8.83-ish baselines (VadCLIP 6.68), per Table 1.
- Coarse-grained XD-Violence AP 86.52 and UCF-Crime AUC 88.29 (Table 3).
- Fine-grained XD-Violence AVG mAP 28.23 vs VadCLIP 24.70 (Table 2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Ex-VAD's fine-grained detection approach | UCF-Crime fine-grained WSVAD · UCF-Crime | mAP@IoU AVG: 6.68 -> 10.15 | Helped |
| Add Ex-VAD's fine-grained detection approach | XD-Violence fine-grained WSVAD · XD-Violence | mAP@IoU AVG: 24.70 -> 28.23 | Helped |
| Add Ex-VAD's coarse-grained detector | XD-Violence · XD-Violence | AP: 88.2 -> 86.52 | Hurt |
| Add Ex-VAD's coarse-grained detector | UCF-Crime · UCF-Crime | AUC: 88.58 -> 88.29 | Hurt |
| Add frame captions without caption cleaning or video-level explanation (ablation) | Ex-VAD coarse-grained detector · UCF-Crime | AUC: 86.76 -> 86.33 | Hurt |
| Add cleaned video-level anomaly explanations instead of frame captions (ablation) | Ex-VAD coarse-grained detector · UCF-Crime | AUC: 86.33 -> 87.86 | Helped |
| Add LAAM label augmentation and alignment (ablation) | Ex-VAD coarse-grained detector · UCF-Crime | AUC: 87.86 -> 88.29 | Helped |
| Use frame captions instead of video-level explainable text (ablation) | Ex-VAD fine-grained detector · UCF-Crime | mAP@IoU AVG: 10.15 -> 10.88 | Helped |
| Add label-augment prompts instead of manually defined [CLS] prompts (ablation) | Ex-VAD fine-grained detector · UCF-Crime | mAP@IoU AVG: 7.87 -> 10.15 | Helped |
| Add label-augment prompts instead of learnable prompts (ablation) | Ex-VAD fine-grained detector · UCF-Crime | mAP@IoU AVG: 8.40 -> 10.15 | Helped |
| Select top 4 semantically similar label phrases (ablation) | Ex-VAD · UCF-Crime | AUC: 87.57 -> 88.28 | Helped |
| Increase selected label phrases from k=4 to k=5 (ablation) | Ex-VAD · UCF-Crime | AUC: 88.28 -> 88.21 | Hurt |
Surprising findings
- Raw frame-level captions alone gave higher fine-grained AVG mAP (10.88) than the LLM-generated explanation text (10.15) on UCF-Crime, though the explanations were chosen for interpretability.
Limitations
- Performance on UCF-Crime coarse-grained AUC is below TCVADS (88.29 vs 88.58), per the authors.
- Excessive label enhancement (k > 5) introduces noise and degrades performance.
- Raw captions outperform explanation text on fine-grained detection, so the explanation module trades some accuracy for interpretability.
Experimental setup
Datasets: UCF-Crime XD-Violence
Benchmarks: UCF-Crime XD-Violence
Metrics: frame-level AUC frame-level AP mAP@IoU 0.1-0.5 AVG mAP
Baselines: RealAD RTFM AVVD DMU CLIP-TSA UMIL VadCLIP STPrompt LAVAD VERA VADor
Builds on: LAVAD VadCLIP STPrompt PEL4VAD LGT-Adapter
Closest papers by meaning
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.92)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.90)
- VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models (CVPR 2025 · similarity 0.89)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.89)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.88)
- MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.