Research map / Video / Video segmentation and tracking
Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a lightweight CvT-based masked auto-encoder for frame-level video anomaly detection, using motion-gradient token weighting, self-distillation with a shared encoder and student decoder, and synthetic UBnormal anomaly augmentation, running at 1655 FPS.
Problem
Video anomaly detection needs to be both accurate and fast for large-scale surveillance, but state-of-the-art methods rely on costly object detectors that run at only about 20-30 FPS, and reconstruction-based AEs often generalize too well to anomalies.
Approach
A masked AE applied to whole frames with (i) reconstruction loss weighted by token-level motion gradient magnitude (plus anomaly maps), (ii) a shared-encoder teacher-student decoder self-distillation whose discrepancy is used as an anomaly score, and (iii) synthetic UBnormal anomalies superimposed on normal training videos with a classification head to detect them.
Key results
- Achieves 91.3% micro AUC on Avenue, the best among frame-level methods, while running at 1655 FPS.
- Reaches 95.4% micro AUC on UCSD Ped2 and 58.5% micro AUC on UBnormal; the ablation shows self-distillation and motion weighting each lift performance.
- Model has about 3M parameters and 0.8 GFLOPs, reported as 8 to 70 times faster than competing methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add motion-gradient token weights (ablation) | Masked AE · Avenue | micro AUC (%): 84.0 -> 84.8 | Helped |
| add self-distillation (ablation) | Masked AE · Avenue | micro AUC (%): 84.8 -> 88.5 | Helped |
| add anomaly-map prediction (ablation) | Masked AE with synthetic data · Avenue | micro AUC (%): 88.5 -> 90.5 | Helped |
| add a classification head (ablation) | Masked AE with anomaly-map prediction · Avenue | micro AUC (%): 90.5 -> 91.3 | Helped |
| add synthetic anomaly data (ablation) | Masked AE · ShanghaiTech | micro AUC (%): 76.3 -> 77.0 | Helped |
| use teacher reconstruction error plus teacher-student discrepancy instead of teacher alone (ablation) | Masked AE output strategy · Avenue | micro AUC (%): 84.0 -> 88.5 | Helped |
| combine teacher, student, and teacher-student discrepancy (ablation) | Masked AE output strategy · Avenue | micro AUC (%): 88.5 -> 86.9 | Hurt |
| use 25% synthetic training data rather than 0% (ablation) | Self-Distilled Masked AE · Avenue | micro AUC (%): 88.5 -> 91.3 | Helped |
| increase synthetic training data from 25% to 50% (ablation) | Self-Distilled Masked AE · Avenue | micro AUC (%): 91.3 -> 90.6 | Hurt |
| increase synthetic training data from 25% to 75% (ablation) | Self-Distilled Masked AE · ShanghaiTech | micro AUC (%): 79.1 -> 77.7 | Hurt |
Surprising findings
- Combining the teacher reconstruction error with the teacher-student discrepancy works better than using the student alone or adding student output to the score, and the synthetic-anomaly proportion peaks at a moderate 25% rather than the maximum.
Limitations
- Micro AUC on UBnormal (58.5%) and ShanghaiTech (79.1%) lags object-centric methods.
- Object-centric methods remain more accurate on Avenue and UCSD Ped2 (e.g., 3.9% below Liu et al. on Ped2 micro AUC).
Experimental setup
Datasets: Avenue ShanghaiTech UBnormal UCSD Ped2
Benchmarks: Avenue ShanghaiTech UBnormal UCSD Ped2
Metrics: micro AUC macro AUC FPS GFLOPs #parameters
Baselines: Georgescu et al. [26] Georgescu et al. [27] Liu et al. [49] Park et al. [60] Gong et al. [28] Ionescu et al. [10] Wang et al. [84]
Builds on: Masked Autoencoders (MAE) Convolutional vision Transformer (CvT) Self-distillation UBnormal
Closest papers by meaning
- Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video (CVPR 2019 · similarity 0.90)
- Anomaly Detection in Video via Self-Supervised and Multi-Task Learning (CVPR 2021 · similarity 0.87)
- Track Any Anomalous Object:A Granular Video Anomaly Detection Pipeline (CVPR 2025 · similarity 0.85)
- Learning Event Completeness for Weakly Supervised Video Anomaly Detection (ICML 2025 · similarity 0.84)
- Extending Video Masked Autoencoders to 128 frames (NeurIPS 2024 · similarity 0.84)
- Noise-Resistant Video Anomaly Detection via RGB Error-Guided Multiscale Predictive Coding and Dynamic Memory (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.