Research map / Video / Video segmentation and tracking
Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MA-PDM, a patch-based conditional diffusion model that predicts the next video frame from appearance (first frame, memory-refined) and motion (frame-difference) conditions, scoring anomalies by reconstruction error.
Problem
One-class video anomaly detection via future frame prediction, where small abnormal objects are missed by frame-level feature prediction and existing AE/diffusion methods ignore combined appearance and motion cues.
Approach
A patch-based diffusion model that denoises random crops conditioned on an appearance encoder with a patch-level memory bank and a temporal-difference motion condition, fused via adaptive layer norm and ControlNet-style skip connections, with sliding-window inference.
Key results
- MA-PDM reaches 98.6 AUC on Ped2, 91.3 on Avenue, 79.2 on ShanghaiTech and 63.4 on UBnormal, the best reported in the comparison table.
- Versus FPDM, gains are reported as 1.2% on Avenue, 0.5% on ShanghaiTech and 0.7% on UBnormal.
- Temporal difference motion gives 91.3 Avenue AUC at 45 FPS versus 88.7 AUC at 25 FPS for optical flow (RAFT).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MA-PDM instead of the strongest listed baseline | Video anomaly detection · Ped2 | AUC (%): 98.1 -> 98.6 | Helped |
| use MA-PDM instead of the strongest listed baseline | Video anomaly detection · Avenue | AUC (%): 91.3 -> 91.3 | No clear effect |
| use MA-PDM instead of the strongest listed baseline | Video anomaly detection · Shanghai | AUC (%): 79.1 -> 79.2 | Helped |
| use MA-PDM instead of the strongest listed baseline | Video anomaly detection · UBnormal | AUC (%): 62.7 -> 63.4 | Helped |
| add patch diffusion to appearance and motion conditions (ablation) | MA-PDM · Ped2, Avenue, Shanghai, UBnormal | AUC (%): 96.7 -> 98.6; 88.8 -> 91.3; 76.7 -> 79.2; 57.8 -> 63.4 | Helped |
| add motion condition (ablation) | Patch diffusion + appearance condition · Ped2, Avenue, Shanghai, UBnormal | AUC (%): 97.4 -> 98.6; 89.9 -> 91.3; 77.1 -> 79.2; 58.4 -> 63.4 | Helped |
| use patch diffusion only instead of appearance condition only (ablation) | MA-PDM module comparison · Ped2, Avenue, Shanghai, UBnormal | AUC (%): 95.8 -> 96.8; 86.2 -> 89.2; 72.4 -> 73.9; 55.3 -> 56.3 | Mixed |
| add patch memory instead of no memory (ablation) | MA-PDM · Ped2, Avenue, Shanghai | AUC (%): 98.0 -> 98.6; 89.9 -> 91.3; 77.9 -> 79.2 | Helped |
| replace global memory with patch memory (ablation) | MA-PDM · Ped2, Avenue, Shanghai | AUC (%): 98.6 -> 98.6; 90.2 -> 91.3; 78.7 -> 79.2 | Mixed |
| replace optical flow with temporal difference (ablation) | MA-PDM motion condition · Ped2 and Avenue | AUC (%): 93.7 -> 98.6; 88.7 -> 91.3 | Helped |
| replace optical flow with temporal difference (ablation) | MA-PDM motion condition · Motion extraction | FPS: 25 -> 45(+20) | Helped |
| vary random-crop patch size from 64 to 256 (ablation) | MA-PDM · Ped2 and Avenue | AUC (%): 98.6 -> 96.8; 91.3 -> 88.7 | Hurt |
Surprising findings
- Motion condition alone with the appearance-only (non-patch) model yields little gain on Ped2 and Avenue, while the patch diffusion model alone already gives strong results (e.g., 96.8 on Ped2).
Limitations
- Inference speed is still insufficient for real-time use, as the authors acknowledge.
- Patch size is dataset-dependent (32 best on Ped2, 64 on others), requiring tuning.
- Gains over the best prior methods are modest on several datasets (e.g., 1.2% on Avenue over FPDM).
Experimental setup
Datasets: Ped2 Avenue ShanghaiTech Campus UBnormal
Benchmarks: Ped2 Avenue ShanghaiTech Campus UBnormal
Metrics: frame-level AUC
Baselines: MNAD FPDM MPN AMMC-Net LGN-Net DLAN-AC USTN-DSC AED-MAE MemAE AMC
Builds on: DDIM ControlNet FPDM MNAD
Closest papers by meaning
- Dual Conditioned Motion Diffusion for Pose-Based Video Anomaly Detection (AAAI 2025 · similarity 0.89)
- Correcting Deviations from Normality: A Reformulated Diffusion Model for Multi-Class Unsupervised Anomaly Detection (CVPR 2025 · similarity 0.88)
- MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection (CVPR 2024 · similarity 0.88)
- DiffDVC: Accurate Event Detection for Dense Video Captioning via Diffusion Models (AAAI 2025 · similarity 0.87)
- Multi-Scale Video Anomaly Detection by Multi-Grained Spatio-Temporal Representation Learning (CVPR 2024 · similarity 0.86)
- Track Any Anomalous Object:A Granular Video Anomaly Detection Pipeline (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.