Research map / Vision-language and multimodal / Multimodal LLMs
Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Video Anomaly Reasoning (VAR) with a Perception-to-Cognition CoT, the Vad-Reasoning dataset, and AVA-GRPO (GRPO plus self-verification rewards from temporal trimming) to train Qwen2.5-VL-7B for anomaly reasoning and detection.
Problem
Existing MLLM-based video anomaly detection methods give shallow descriptions without explicit reasoning, and VAD datasets lack structured reasoning annotations; RL training for open-ended anomaly reasoning also lacks clear objectives when only video-level weak labels exist.
Approach
A structured perception-to-cognition CoT (global/local perception, shallow/deep cognition, answer) used for SFT, followed by AVA-GRPO which adds an anomaly verification reward: the video is temporally trimmed based on the predicted anomaly (or the start/end is dropped for normal predictions) and the model's consistency on the trimmed video is rewarded or penalized, plus a length reward.
Key results
- Vad-R1 reaches BLEU-2 0.233 and Acc 0.875 / mIoU 0.713 on the Vad-Reasoning test set, outperforming the open-source video MLLMs and proprietary reasoning models (Gemini2.5-pro, QVQ-Max, o4-mini) on anomaly reasoning BLEU.
- On the ablation of training strategies, Qwen2.5-VL with SFT+AVA-GRPO reaches BLEU-2 0.233 and R@0.5 0.706, versus 0.113 BLEU-2 and 0.563 R@0.5 for the base model.
- On VANE, Vad-R1 outperforms all listed MLLM-based VAD methods on every subset, e.g. 78.00 on OpenSora and 75.76 on Avenue versus 52.00 and 36.36 for HAWK.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use structured P2C-CoT reasoning instead of direct answering (ablation) | Qwen2.5-VL-7B · Vad-Reasoning test | F1: 0.597 -> 0.730 | Helped |
| Use random reasoning instead of direct answering (ablation) | Qwen2.5-VL-7B · Vad-Reasoning test | F1: 0.597 -> 0.540 | Hurt |
| Use structured P2C-CoT reasoning instead of direct answering (ablation) | Qwen3-8B · Vad-Reasoning test | Recall: 0.368 -> 0.681 | Helped |
| Train with structured reasoning rather than only the final answer (ablation) | Vad-R1 · Vad-Reasoning test | BLEU-2: 0.268 -> 0.293 | Helped |
| Add SFT to the Qwen2.5-VL base model (ablation) | Vad-R1 · Vad-Reasoning test | BLEU-2: 0.113 -> 0.219 | Helped |
| Add AVA-GRPO after SFT (ablation) | Vad-R1 · Vad-Reasoning test | mIoU: 0.612 -> 0.713 | Helped |
| Add SFT before AVA-GRPO rather than applying RL to the base model (ablation) | Vad-R1 · Vad-Reasoning test | BLEU-2: 0.143 -> 0.233 | Helped |
| Add the length reward to GRPO (ablation) | Vad-R1 · Vad-Reasoning test, 16 frames | ROUGE-L: 0.502 -> 0.529 | Helped |
| Add the anomaly verification reward to GRPO (ablation) | Vad-R1 · Vad-Reasoning test, 16 frames | Precision: 0.861 -> 0.866 | Helped |
| Combine length and anomaly verification rewards (ablation) | Vad-R1 · Vad-Reasoning test, 16 frames | Precision: 0.861 -> 0.882 | Helped |
| Combine length and anomaly verification rewards (ablation) | Vad-R1 · Vad-Reasoning test, 32 frames | mIoU: 0.695 -> 0.726 | Helped |
| Increase input frames from 16 to 64 at 128 × 28 × 28 max pixels (ablation) | Vad-R1 · Vad-Reasoning test | BLEU-2: 0.233 -> 0.244 | Helped |
Surprising findings
- Random reasoning (unstructured CoT) gives little gain and can underperform direct answering for Qwen2.5-VL and Qwen3-8B, while structured P2C-CoT gives clear gains.
- RL alone without SFT gives only limited improvement, while SFT alone improves BLEU but lowers ROUGE-1 and ROUGE-L.
Limitations
- Evaluation relies partly on proprietary LLM judges and token-overlap metrics (BLEU, METEOR) that the authors acknowledge are limited for semantic reasoning quality.
- Annotations for the SFT subset are generated entirely by proprietary models (Qwen-VL-Max, Qwen-Max), so quality depends on them and the approach relies on closed-source tooling.
- The VANE evaluation excludes the UCF-Crime subset because it overlaps with the training data.
Experimental setup
Datasets: UCF-Crime XD-Violence TAD ShanghaiTech UBnormal ECVA Vad-Reasoning-SFT Vad-Reasoning-RL
Benchmarks: Vad-Reasoning test set VANE
Metrics: BLEU-2 METEOR ROUGE-2 ROUGE-1 ROUGE-L Accuracy F1 Precision Recall mIoU
Baselines: Qwen2.5-VL-7B Qwen3-8B InternVideo2.5 InternVL3 VideoChat-Flash VideoLLaMA3 LLaVA-NeXT-Video Open-R1-Video Video-R1 VideoChat-R1 Holmes-VAD Holmes-VAU
Builds on: GRPO DeepSeek-R1 Video-UTR HAWK
Closest papers by meaning
- VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models (CVPR 2025 · similarity 0.91)
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.90)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.89)
- MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection (NeurIPS 2025 · similarity 0.88)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.88)
- PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.