Research map / Vision-language and multimodal / Multimodal LLMs
VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VERA, which learns guiding questions for a frozen VLM via verbalized learner-optimizer interaction on video-level labels, then scores frames coarse-to-fine for explainable video anomaly detection without instruction tuning.
Problem
Explainable video anomaly detection with VLMs typically needs extra LLM reasoning modules at inference or instruction tuning on fine-grained labeled data, both costly; the paper asks whether a frozen VLM can integrate description and reasoning for VAD without IT.
Approach
Treat a set of natural-language guiding questions as learnable parameters, optimized by verbal feedback between a learner VLM (binary video classification) and an optimizer VLM using only coarse video-level labels; at inference, segment scores from these questions are refined with similarity-based scene retrieval, Gaussian temporal smoothing and position weighting.
Key results
- VERA reaches 86.55% AUC on UCF-Crime, the highest among explainable VAD methods reported (LAVAD 80.28%, Holmes-VAD 84.61%, VADor 85.90%).
- VERA reaches 88.26% AUC on XD-Violence versus 85.36% for LAVAD.
- Ablation: coarse-to-fine scoring raises AUC from 76.10% (initial segment scores) to 86.55%, and learned questions beat manually written ones (81.15%) and no questions (78.81%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scene-context retrieval to the initial segment scores (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 76.10 -> 84.53 (+8.43) | Helped |
| add Gaussian smoothing after retrieval (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 84.53 -> 85.48 (+0.95) | Helped |
| add temporal position weighting after smoothing (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 85.48 -> 86.55 (+1.07) | Helped |
| use uniform rather than random training-frame sampling (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 83.63 -> 86.55 | Helped |
| use uniform rather than TSN training-frame sampling (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 82.63 -> 86.55 | Helped |
| add guiding questions to the VLM prompt (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 78.81 -> 86.55 | Helped |
| replace manually written questions with iteratively learned questions (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 81.15 -> 86.55 | Helped |
| include video frames in the optimizer input when updating questions (ablation) | VERA on UCF-Crime · UCF-Crime | AUC (%): 78.06 -> 86.55 | Helped |
| transfer questions learned by InternVL2-8B to InternVL2-40B rather than use questions learned by InternVL2-40B (ablation) | VERA · UCF-Crime | AUC (%): 86.72 -> 85.24 | Hurt |
| transfer questions learned by InternVL2-40B to InternVL2-8B rather than use questions learned by InternVL2-8B (ablation) | VERA · UCF-Crime | AUC (%): 86.55 -> 80.43 | Hurt |
| replace general learned questions with traffic-specific questions (ablation) | VERA · UCF-Crime Traffic | AUC (%): 70.43 -> 67.53 | Hurt |
| replace general learned questions with store-specific questions (ablation) | VERA · UCF-Crime Store | AUC (%): 72.58 -> 44.84 | Hurt |
Surprising findings
- On XD-Violence AP, VERA (70.54%) trails methods trained on full training frames such as Holmes-VAD (84.96%), which the authors attribute to using only 0.19% of training frames.
- Questions learned with InternVL2-8B transfer to InternVL2-40B but not vice versa, and transfer across VLM architectures shows a notable gap.
Limitations
- AP on XD-Violence lags methods that train on full frames.
- Learned questions transfer poorly across VLM architectures and from larger to smaller models.
- Optimization is noisy and validation accuracy oscillates during verbal training.
Experimental setup
Datasets: UCF-Crime XD-Violence
Benchmarks: UCF-Crime XD-Violence
Metrics: frame-level AUC AP
Baselines: LAVAD Holmes-VAD VADor CLIP-TSA ZS CLIP ZS ImageBind LLaVA-1.5 RTFM MGFN OVVAD
Builds on: verbalized machine learning TextGrad LAVAD
Closest papers by meaning
- Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought (NeurIPS 2025 · similarity 0.91)
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.91)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.89)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.89)
- Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models (ICML 2025 · similarity 0.89)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.