Research map / Video / Video segmentation and tracking
Prompt-Enhanced Multiple Instance Learning for Weakly Supervised Video Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes prompt-enhanced MIL for weakly supervised video anomaly detection, using learnable abnormal-class text prompts and a normal context prompt to improve diverse abnormal pattern modeling and boundary clarity.
Problem
Weakly supervised video anomaly detection with only video-level labels struggles to model diverse abnormal patterns with binary supervision and to produce clear event boundaries because abnormalities are coupled with their context.
Approach
Abnormal-aware prompts combine abnormal class text embeddings with learnable prompt tokens and are aligned fine-grainedly with video features via event relevance reasoning; a normal context prompt, learned in a second stage with a frozen model, is concatenated at inference to enrich ambiguous context snippets and sharpen event boundaries.
Key results
- On XD-Violence with I3D+VGGish features, the method reaches 88.21 AP, a +2.54 gain over the prior best HyperVD (85.67).
- On ShanghaiTech with I3D RGB, AUC of 98.35 (+0.87 over the prior best).
- On UCF-Crime, AUC of 86.83 (+0.73 over UMIL's 86.75), with smaller gains attributed to low coupling between abnormality and context.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PE-MIL with I3D RGB + VGGish instead of HyperVD | XD-Violence anomaly detection · XD-Violence test | AP: 85.67 -> 88.21 | Helped |
| Use PE-MIL with I3D RGB instead of CRFD | ShanghaiTech anomaly detection · ShanghaiTech test | AUC: 97.48 -> 98.35 | Helped |
| Use PE-MIL with I3D RGB instead of UMIL | UCF-Crime anomaly detection · UCF-Crime test | AUC: 86.75 -> 86.83 | Helped |
| Add Scale-Aware Prediction Head (ablation) | PE-MIL baseline · XD-Violence | AP: 78.73 -> 80.82 | Helped |
| Add Abnormal-Aware Prompt Learning (ablation) | PE-MIL with SA-Head · XD-Violence | AP: 80.82 -> 86.62 | Helped |
| Add Normal Context Prompt (ablation) | PE-MIL with SA-Head and APL · XD-Violence | AP: 86.62 -> 88.21 | Helped |
| Add Event Relevance Reasoning instead of constant target factors (ablation) | APL · XD-Violence | AP: 85.61 -> 87.78 | Helped |
| Add learnable prompt without Prompt Constraint loss (ablation) | APL · XD-Violence | AP: 87.78 -> 87.45 | Hurt |
| Add Prompt Constraint loss with the learnable prompt (ablation) | APL · XD-Violence | AP: 87.45 -> 88.21 | Helped |
| Vary NCP length from 0 through 50 (ablation) | PE-MIL · XD-Violence | AP: 86.62 87.47 87.87 88.09 88.21 88.20 | Helped |
Surprising findings
- On UCF-Crime the gains are small, which the authors attribute to homogeneous abnormal events and fixed-view surveillance with low abnormality-context coupling.
Limitations
- Gains on UCF-Crime are marginal compared with other datasets.
- Relies on abnormal class annotations to build abnormal-aware prompts.
Experimental setup
Datasets: UCF-Crime ShanghaiTech XD-Violence
Benchmarks: XD-Violence ShanghaiTech UCF-Crime
Metrics: AP AUC
Baselines: MIL-Rank RTFM MSL MGFN UR-DMU CoMo PEL4VAD HyperVD UMIL S3R
Builds on: multiple instance learning ranking loss CoMo PEL4VAD MSL
Closest papers by meaning
- Federated Weakly Supervised Video Anomaly Detection with Multimodal Prompt (AAAI 2025 · similarity 0.91)
- Learning Event Completeness for Weakly Supervised Video Anomaly Detection (ICML 2025 · similarity 0.90)
- TLMA: Mitigating the Impact of Weakly Labeled Information for Video Anomaly Detection (CVPR 2026 · similarity 0.90)
- Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection (CVPR 2024 · similarity 0.90)
- Towards Multi-Domain Learning for Generalizable Video Anomaly Detection (NeurIPS 2024 · similarity 0.89)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.