Research map / Video / Video-language understanding
MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoniTor, a training-free online video anomaly detection framework that feeds streaming BLIP-2 captions into GLM-4-Flash with LSTM-inspired memory gating, a scoring queue, anomaly priors, and next-frame behavior prediction.
Problem
Video anomaly detection in online (streaming, frame-by-frame) settings has received little attention because of real-time constraints and computational cost; existing LLM-based VAD methods are offline and sensitive to prompts and historical noise.
Approach
Combines a VLM captioning pipeline with an LLM scorer that is guided by an LSTM-style dual memory (long-term with similarity-based forgetting gate, short-term over the last two frames), a queue of recent captions indexed by anomaly score, Wikipedia-derived anomaly definitions, and a next-frame prediction step, followed by temporal score smoothing.
Key results
- MoniTor reaches 82.57% AUC on UCF-Crime, versus 76.06% for online-LAVAD and 80.28% for offline LAVAD, with no training.
- On XD-Violence MoniTor achieves 55.01% AP and 79.11% AUC, improving over online-LAVAD (52.63 AP, 76.01 AUC).
- Decision period of 0.6 s with 5.9 s processing time and 6.5 s delay, compared with REWARD's 6.4 s decision period and 6.9 s delay.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the Standard Scoring Queue (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 76.06 -> 78.65 | Helped |
| Add Anomaly Priors Integration (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 76.06 -> 77.85 | Helped |
| Add Dynamic Memory Gating Module (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 76.06 -> 78.88 | Helped |
| Add Behavior Prediction and Dynamic Analysis (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 76.06 -> 78.30 | Helped |
| Add weight assignment (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 76.06 -> 77.02 | Helped |
| Add long-term memory, short-term memory, and the forgetting gate (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 78.27 -> 78.88 | Helped |
| Add the forgetting gate to long-term memory (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 78.27 -> 78.66 | Helped |
| Use Encyclopædia Britannica instead of Wikipedia for anomaly priors (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 77.85 -> 77.91 | Helped |
| Use World Book instead of Wikipedia for anomaly priors (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 77.85 -> 77.42 | Hurt |
| Use domain expert explanations instead of Wikipedia for anomaly priors (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC: 77.85 -> 78.39 | Helped |
| Initialize with random generic patterns rather than domain-specific queue and memory content (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC change: -0.32 | Hurt |
| Prefill the scoring queue from normal videos and anomaly categories (ablation) | MoniTor on UCF-Crime · UCF-Crime | AUC change: +1.98 | Helped |
Surprising findings
- Roughly 60% of detection errors reportedly occur around camera transitions, which disrupt the memory gating module.
- Random initialization of the scoring queue slightly hurts performance (-0.32% AUC).
Limitations
- Camera transitions disrupt memory gating and cause error spikes.
- Frame-level processing lacks temporal context, limiting accuracy relative to video-segment weakly supervised methods.
- Relies on large LLMs and VLMs that are computationally heavy for edge devices.
Experimental setup
Datasets: UCF-Crime XD-Violence Wikipedia anomaly definitions
Benchmarks: UCF-Crime XD-Violence
Metrics: AUC AP decision period processing time delay
Baselines: online-LAVAD REWARD LAVAD EventVAD Video-Llama2 LLAVA-1.5 MOVAD
Builds on: LAVAD LSTM REWARD
Closest papers by meaning
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.92)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.90)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.89)
- Advancing Video Anomaly Detection: A Concise Review and a New Dataset (NeurIPS 2024 · similarity 0.89)
- Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought (NeurIPS 2025 · similarity 0.88)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.