Research map / Video / Video segmentation and tracking
Anomize: Better Open Vocabulary Video Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Anomize, a frozen-CLIP framework for open vocabulary video anomaly detection that uses a text-augmented dual stream for detection and group-guided GPT-4-generated label descriptions for categorization.
Problem
Open vocabulary video anomaly detection must detect and categorize novel anomalies, which suffers from detection ambiguity (unfamiliar anomalies get unreliable scores) and categorization confusion (novel anomalies are misclassified as visually similar base classes).
Approach
Augment visual features with text in two complementary streams (dynamic: LSTM temporal features plus label descriptions; static: CLIP features plus a GPT-generated concept library), and encode labels with group-based GPT-4 descriptions so that novel labels align with visually similar base labels.
Key results
- On XD-VIOLENCE, AP improves from 66.53 to 69.31 over Wu et al. OVVAD, with novel-case AP_n 84.24 vs 76.03.
- Categorization Top-1 accuracy on XD-VIOLENCE rises from 64.68 to 90.29, and on UCF-CRIME from 41.43 to 47.14.
- On UCF-CRIME, detection AUC is 84.49, below the 86.40 of Wu et al. OVVAD, so the method is only competitive there.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add group-guided text encoding (ablation) | Anomize categorization · XD-VIOLENCE anomalous videos | Top-1 accuracy: 53.42 -> 56.07 | Helped |
| add group-guided text encoding (ablation) | Anomize categorization · UCF-CRIME anomalous videos | Top-1 accuracy: 25.71 -> 27.86 | Helped |
| add feature fusion (ablation) | Anomize categorization · XD-VIOLENCE anomalous videos | Top-1 accuracy: 53.42 -> 56.51 | Helped |
| add separation loss Lsep (ablation) | Anomize categorization · XD-VIOLENCE anomalous videos | Top-1 accuracy: 89.95 -> 90.29 | Helped |
| add separation loss Lsep (ablation) | Anomize categorization · UCF-CRIME anomalous videos | Top-1 accuracy: 46.43 -> 47.14 | Helped |
| remove segmented training (ablation) | Anomize categorization · UCF-CRIME anomalous videos | Top-1 accuracy: 46.43 -> 47.14 | No clear effect |
| add text augmentation to the static stream (ablation) | Anomize detection · XD-VIOLENCE | AP: 54.75 -> 59.65 | Helped |
| integrate static and dynamic streams (ablation) | Anomize detection · XD-VIOLENCE | AP: 59.26 -> 69.31 | Helped |
| add loss weight wi to integrated streams (ablation) | Anomize detection · XD-VIOLENCE | AP: 58.22 -> 69.31 | Helped |
| use Anomize instead of Wu et al. (2023) | OVVAD detection · XD-VIOLENCE | AP: 66.53 -> 69.31 | Helped |
| use Anomize instead of Wu et al. (2023) | OVVAD detection · UCF-CRIME | AUC: 86.40 -> 84.49 | Hurt |
| use Anomize instead of Wu et al. (2023) | OVVAD categorization · XD-VIOLENCE anomalous videos | Top-1 accuracy: 64.68 -> 90.29 | Helped |
Surprising findings
- Adding a temporal encoder alone degrades novel-case categorization, which motivates the guided encoding.
- On UCF-CRIME, detection is not the best among compared methods, which the authors attribute to the lightweight temporal encoder and segmented training.
Limitations
- UCF-CRIME detection AUC trails the prior OVVAD method.
- Dependence on GPT-4 generated groups and descriptions for label encoding.
Experimental setup
Datasets: XD-VIOLENCE UCF-CRIME
Benchmarks: XD-VIOLENCE UCF-CRIME
Metrics: AP AUC Top-1 accuracy
Baselines: Zhu et al. Sultani et al. Wu et al. (XD-Violence) RTFM Wu et al. OVVAD
Builds on: CLIP VadCLIP Wu et al. OVVAD (open vocabulary video anomaly detection) multiple instance learning
Closest papers by meaning
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.87)
- Uncovering What Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly (CVPR 2024 · similarity 0.86)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.85)
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.83)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.83)
- Track Any Anomalous Object:A Granular Video Anomaly Detection Pipeline (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.