Research map / Video / Video segmentation and tracking
Multi-Scale Video Anomaly Detection by Multi-Grained Spatio-Temporal Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes multi-grained spatio-temporal self-supervised proxy tasks (continuity judgment, discontinuity localization, and contrastive missing-frame estimation in feature space) for multi-scale video anomaly detection.
Problem
Video anomaly detection methods perform poorly on small-scale localized anomalies because RGB reconstruction/prediction is dominated by background noise and coarse-grained modeling lacks discriminative fine-grained features.
Approach
Uses video continuity to build three jointly optimized proxy tasks at coarse and fine granularity, and replaces RGB frame reconstruction with a contrastive missing-frame estimation task in feature space, with frame-level PSNR-based anomaly scoring.
Key results
- Achieves 87.5% Micro AUC and 93.0% Macro AUC on ShanghaiTech with multi-task backbone, and 85.1% Micro AUC without it.
- Reaches 94.3% Micro AUC and 94.5% Macro AUC on Avenue with multi-task backbone and virtual data.
- Improves UCF-Crime Micro AUC to 80.6% (5.1 points over second best) and Campus Micro AUC to 67.3% without scene-conditioned backbone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use all three proxy tasks jointly (ablation) | I3D-RGB VAD framework · ShanghaiTech | Micro AUC: 70.2 -> 85.1 | Helped |
| Add continuity judgment (ablation) | I3D-RGB VAD framework · ShanghaiTech | Micro AUC: 70.2 -> 77.2 | Helped |
| Add discontinuity localization (ablation) | I3D-RGB VAD framework · ShanghaiTech | Micro AUC: 70.2 -> 83.4 | Helped |
| Add missing frame estimation (ablation) | I3D-RGB VAD framework · Campus | Micro AUC: 58.1 -> 65.2 | Helped |
| Replace frame prediction with missing frame estimation (ablation) | I3D-RGB VAD framework · ShanghaiTech | Micro AUC: 84.2 -> 85.1 | Helped |
| Set input clip length to T=8 instead of T=12 (ablation) | Overall framework · ShanghaiTech | Micro AUC: 85.1 -> 84.2 | Hurt |
| Set input clip length to T=16 instead of T=12 (ablation) | Overall framework · ShanghaiTech | Micro AUC: 85.1 -> 83.9 | Hurt |
| Add the three proxy tasks using a multi-task backbone and virtual data | Ours · ShanghaiTech | Micro AUC: 85.0 -> 87.5 | Helped |
| Use the proposed method with a scene-conditioned backbone | Ours · Campus | Micro AUC: 68.2 -> 70.1 | Helped |
| Use the proposed method | Video anomaly detection · UCF-Crime | Micro AUC: 75.5 -> 80.6 | Helped |
Surprising findings
- Continuity judgment alone at T=8 is competitive on ShanghaiTech, but performance drops with longer clips for that task.
Limitations
- Relies on multi-task backbones and virtual data from prior work to reach best numbers on Avenue and ShanghaiTech.
- Sensitive to input clip length T; performance peaks at T=12 and degrades at other lengths.
Experimental setup
Datasets: Avenue ShanghaiTech UCF-Crime Campus
Benchmarks: Avenue ShanghaiTech UCF-Crime Campus
Metrics: Micro AUC Macro AUC
Baselines: PMem RoadMap DEDD-Net USTN-DSC Georgescu et al. (CVPR21) Liu et al. (ECCV22 / TPAMI) Cao et al. (scene-conditioned) Acsintoae et al. Ristea et al. Sun et al. (CVPR23)
Builds on: frame prediction object-centric video anomaly detection (Georgescu et al.) I3D triplet loss video continuity (Wang et al.)
Closest papers by meaning
- Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model (AAAI 2025 · similarity 0.86)
- MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection (CVPR 2024 · similarity 0.86)
- Noise-Resistant Video Anomaly Detection via RGB Error-Guided Multiscale Predictive Coding and Dynamic Memory (CVPR 2025 · similarity 0.86)
- Track Any Anomalous Object:A Granular Video Anomaly Detection Pipeline (CVPR 2025 · similarity 0.85)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.85)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.