Research map / Video / Video-language understanding
Harnessing Large Language Models for Training-free Video Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAVAD, a training-free video anomaly detection method that uses VLM captions, LLM-based temporal summarization and anomaly scoring, and VLM cross-modal similarity to clean captions and refine scores.
Problem
Video anomaly detection normally requires training on labeled or unlabeled video data, which limits generalization across domains and is costly due to data collection; the paper asks whether VAD can be done without any training.
Approach
Decomposes VAD into captioning, LLM-based temporal summarization and anomaly scoring, plus cross-modal (ImageBind/CLIP-style) caption cleaning and video-text score refinement, using only frozen pre-trained models at inference time.
Key results
- LAVAD achieves 80.28% AUC on UCF-Crime, a +0.52% gain over the best unsupervised method DyAnNet (79.76%) and +6.08% over GCL.
- On XD-Violence, LAVAD reaches 62.01% AP and 85.36% AUC, outperforming RareAnom's 68.33% AUC by +17.03 points in AUC ROC (the paper's stated margin).
- Ablation on UCF-Crime shows removing caption cleaning drops AUC to 76.48, removing temporal summary to 72.70, and removing score refinement to 72.79 versus 80.28 for the full method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LAVAD instead of DyAnNet | UCF-Crime video anomaly detection · UCF-Crime test | AUC: +0.52% | Helped |
| use LAVAD instead of LLaVA-1.5 | UCF-Crime video anomaly detection · UCF-Crime test | AUC | Helped |
| use LAVAD instead of RareAnom | XD-Violence video anomaly detection · XD-Violence test | AUC: +17.03% | Helped |
| use LAVAD instead of LLaVA-1.5 | XD-Violence video anomaly detection · XD-Violence test | AP | Helped |
| use LAVAD instead of LLaVA-1.5 | XD-Violence video anomaly detection · XD-Violence test | AUC | Helped |
| add image-text caption cleaning (ablation) | LAVAD · UCF-Crime test | AUC: −3.8% | Helped |
| add LLM-based temporal summary anomaly scoring (ablation) | LAVAD · UCF-Crime test | AUC: −7.58% | Helped |
| add video-text score refinement (ablation) | LAVAD · UCF-Crime test | AUC: −7.49% | Helped |
| add an anomaly prior to the context prompt (ablation) | LAVAD · UCF-Crime test | AUC: 79.32 -> 79.38 | No clear effect |
| add impersonation to the context prompt (ablation) | LAVAD · UCF-Crime test | AUC: +0.96% | Helped |
| add both anomaly prior and impersonation to the context prompt (ablation) | LAVAD · UCF-Crime test | AUC: 79.32 -> 79.77 | Mixed |
| increase the number of semantically similar temporal summaries used for score refinement (ablation) | LAVAD · UCF-Crime test | AUC | Helped |
Surprising findings
- Naive zero-shot VLMs (CLIP, ImageBind) perform near chance on UCF-Crime (53-56% AUC), and even LLaVA-1.5 direct per-frame scoring reaches only 72.84%.
- Including both anomaly prior and impersonation in the context prompt did not improve over impersonation alone.
Limitations
- Performance remains below fully supervised and weakly supervised trained methods (e.g. up to 87.58% AUC on UCF-Crime).
- Relies on noisy frame-level captions from BLIP-2 and on thresholding for anomaly localization.
Experimental setup
Datasets: UCF-Crime XD-Violence
Benchmarks: UCF-Crime XD-Violence
Metrics: AUC ROC AP
Baselines: ZS CLIP ZS ImageBind LLaVA-1.5 DyAnNet RareAnom GCL RTFM MSL
Builds on: BLIP-2 ImageBind Llama-2
Closest papers by meaning
- MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection (NeurIPS 2025 · similarity 0.92)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.92)
- Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models (ICML 2025 · similarity 0.92)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.92)
- VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models (CVPR 2025 · similarity 0.91)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.