Research map / Video / Video-language understanding
A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free, zero-shot chained test-time reasoning framework over frozen VLMs/LLMs that links video anomaly detection, localization, and understanding.
Problem
Video anomaly analysis stops at frame-wise detection or relies on dataset-specific supervision; existing methods do not provide holistic temporal, spatial, and textual analysis without training.
Approach
Chain intra-task reasoning (score-based surrogate video probability, tag extraction from the most suspicious window, and a score-gated second reasoning pass) with inter-task chaining that feeds tags and scores into spatial localization and textual understanding prompts, all with frozen VLM/LLMs.
Key results
- Zero-shot VAD reaches 84.28% AUC on UCF-Crime and 91.34% AUC on XD-Violence, outperforming the best prior zero-shot method LAVAD (80.28% and 85.36%).
- Ablation shows the full pipeline improves UCF-Crime AUC from 77.67% (direct VLM query) to 84.28%, a gain of 6.61 points.
- Using extracted tags t_V as priors matches or exceeds ground-truth class names for VAD (84.28% vs 83.91% AUC on UCF-Crime).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add score-gated reasoning to LLM scoring and prior reasoning (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 77.40 -> 84.28 | Helped |
| Add prior reasoning without score-gated reasoning to LLM scoring (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 77.67 -> 77.40 | Hurt |
| Add LLM scoring while omitting prior reasoning and score-gated reasoning (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 77.67 -> 80.38 | Helped |
| Add automatically extracted anomaly tags to the VAD prompt (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 81.86 -> 84.28 | Helped |
| Add oracle anomaly class tags to the VAD prompt (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 81.86 -> 83.91 | Helped |
| Add tag prompt refinement to InterTC; leave out bounding-box overlay (ablation) | InterTC VAU · UCF-Crime | BLEU: 0.2147 -> 0.3328 | Helped |
| Add bounding-box overlay after tag prompt refinement (ablation) | InterTC VAU · UCF-Crime | BLEU: 0.3328 -> 0.3453 | Helped |
| Add tag prompt refinement to InterTC; leave out bounding-box overlay (ablation) | InterTC VAU · XD-Violence | BLEU: 0.2897 -> 0.3857 | Helped |
| Add bounding-box overlay after tag prompt refinement (ablation) | InterTC VAU · XD-Violence | BLEU: 0.3857 -> 0.3993 | Helped |
| Increase the reasoning-gate margin from m=0.05 to m=0.40 (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 84.28 -> 79.21 | Hurt |
| Change suspicious window length from max(300,T/10) to max(300,T/5) (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 84.28 -> 81.07 | Hurt |
| Replace VideoLLaMA3-7B with VideoLLaMA3-2B in the vision backbone (ablation) | IntraTR VAD · UCF-Crime | AUC (%): 84.28% -> 83.35% | Hurt |
Surprising findings
- Extracted anomaly tags t_V outperformed ground-truth oracle class names as VAD priors on UCF-Crime.
- Unrestricted LLM scoring with prior reasoning on all samples decreased AUC slightly (77.40% vs 77.67% baseline), attributed to over-thinking noise.
Limitations
- Performance is bounded by the frozen MLLMs' prior knowledge and may inherit pretraining biases.
- Reduced sensitivity to highly subtle or domain-specific anomalies compared with fine-tuned models.
- Uniform sampling may miss very short events; generated descriptions can be overly verbose.
Experimental setup
Datasets: UCF-Crime XD-Violence UBnormal MSAD HIVAU-70k
Benchmarks: UCF-Crime XD-Violence UBnormal MSAD HIVAU-70k
Metrics: AUC AP TIoU BLEU CIDEr METEOR ROUGE GPT-R GPT-D GPT-C
Baselines: LAVAD GLM-4.1V-9B-Thinking CLIP LLaVA-1.5 Holmes-VAU Hawk VERA AnomalyRuler VideoLLaMA3-7B + Llama3.1-8B VideoChat-Flash-2B InternVideo2.5-8B
Builds on: LAVAD
Closest papers by meaning
- VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models (CVPR 2025 · similarity 0.88)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.88)
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- HAWK: Learning to Understand Open-World Video Anomalies (NeurIPS 2024 · similarity 0.87)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.87)
- Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.