Research map / Adversarial robustness and security / Attacks on text and graphs
FrameShield: Adversarially Robust Video Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FrameShield, a two-stage adversarial training framework for weakly supervised video anomaly detection that uses PromptMIL pseudo-labels and synthetic Spatiotemporal Region Distortion anomalies to get frame-level adversarial training.
Problem
Weakly supervised video anomaly detectors are highly vulnerable to adversarial perturbations, and standard adversarial training is ineffective because only video-level labels are available, so frame-level predictions lack supervision.
Approach
Attack per-frame/chunk anomaly scores (raising normal scores, lowering abnormal ones) instead of MIL video-level loss, and make adversarial training feasible by creating frame-level labels via a PromptMIL pseudo-labeler on real anomalies and SRD synthetic pseudo-anomalies with exact labels.
Key results
- Under PGD-1000 with eps=0.5/255, FrameShield attains overall AUROC of 77.2 (TAD), 78.7 (UCF Crime), 76.2 (MSAD), while baselines such as RTFM, TEVAD, MGFN drop to roughly 0-12 under PGD.
- On abnormal-only AUROC under PGD, FrameShield reaches 30.0 (TAD), 53.4 (UCF Crime), 60.2 (MSAD), versus 3.6-13.3 for the baselines.
- Clean performance is slightly reduced relative to baselines (e.g., TAD clean AUC_O 85.1 vs 92.9 for UMIL), which the authors frame as an acceptable robustness trade-off.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adversarially train with frame-level supervision, pseudo labels, and SRD pseudo-anomalies | FrameShield · UCSD Ped2, PGD-1000 overall test set | AUROC: 7.3 -> 81.3 | Helped |
| adversarially train with frame-level supervision, pseudo labels, and SRD pseudo-anomalies | FrameShield · Shanghai, PGD-1000 overall test set | AUROC: 11.2 -> 87.1 | Helped |
| adversarially train with frame-level supervision, pseudo labels, and SRD pseudo-anomalies | FrameShield · TAD, PGD-1000 overall test set | AUROC: 9.5 -> 77.2 | Helped |
| adversarially train with frame-level supervision, pseudo labels, and SRD pseudo-anomalies | FrameShield · UCF Crime, PGD-1000 overall test set | AUROC: 11.8 -> 78.7 | Helped |
| adversarially train with frame-level supervision, pseudo labels, and SRD pseudo-anomalies | FrameShield · MSAD, PGD-1000 overall test set | AUROC: 12.1 -> 76.2 | Helped |
| combine pseudo-label supervision with pseudo-anomaly supervision rather than using either alone (ablation) | FrameShield · TAD, PGD-1000 overall test set | AUROC: 70.3 / 73.2 -> 77.2 | Helped |
| combine pseudo-label supervision with pseudo-anomaly supervision rather than using either alone (ablation) | FrameShield · UCF Crime, PGD-1000 overall test set | AUROC: 68.2 / 71.3 -> 78.7 | Helped |
| combine pseudo-label supervision with pseudo-anomaly supervision rather than using either alone (ablation) | FrameShield · MSAD, PGD-1000 overall test set | AUROC: 62.0 / 64.2 -> 76.2 | Helped |
| use Grad-CAM foreground selection instead of random-region selection (ablation) | SRD · TAD, PGD-1000 overall test set | AUROC: 66.2 -> 77.2 | Helped |
| use Grad-CAM foreground selection instead of random-region selection (ablation) | SRD · Shanghai, PGD-1000 overall test set | AUROC: 78.3 -> 87.1 | Helped |
| use Grad-CAM foreground selection instead of random-region selection (ablation) | SRD · MSAD, PGD-1000 overall test set | AUROC: 65.9 -> 76.2 | Helped |
| add curved-vector temporal motion to SRD (ablation) | SRD · TAD, PGD-1000 overall test set | AUROC: 53.2 / 64.1 -> 77.2 | Helped |
Surprising findings
- Training with larger epsilon (1/255 or 2/255) under fully supervised Shanghai labels collapses clean performance to near-random, suggesting perturbation magnitude is very sensitive.
- Video-level attacks on clean MIL models with max, LSE, SmoothMax, or ABMIL aggregators fail to drive AUC to zero, whereas frame-level attacks do.
Limitations
- Clean-data detection performance drops relative to strong non-robust baselines.
- Grad-CAM from ResNet18 is an imperfect foreground detector, as acknowledged by the authors.
- Adversarial training with MIL loss fails to converge when label noise (false negatives) is present, so the method depends on pseudo-label quality.
Experimental setup
Datasets: MSAD UCF-Crime ShanghaiTech TAD UCSD Ped2
Benchmarks: UCSD Ped2 ShanghaiTech TAD UCF-Crime MSAD
Metrics: AUROC (AUC_O) AUROC on abnormal videos (AUC_A)
Baselines: RTFM TEVAD MGFN Base MIL UMIL UR-DMU VAD-CLIP
Builds on: X-CLIP PGD TRADES UMIL Grad-CAM
Closest papers by meaning
- PatchGuard: Adversarially Robust Anomaly Detection and Localization through Vision Transformers and Pseudo Anomalies (CVPR 2025 · similarity 0.87)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.87)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.86)
- Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection (CVPR 2024 · similarity 0.86)
- Learning Event Completeness for Weakly Supervised Video Anomaly Detection (ICML 2025 · similarity 0.86)
- Just Dance with pi! A Poly-modal Inductor for Weakly-supervised Video Anomaly Detection (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.