Research map / Video / Video segmentation and tracking
Track Any Anomalous Object:A Granular Video Anomaly Detection Pipeline
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TAO, a granular video anomaly detection pipeline that turns object-level anomaly detection boxes into SAM2 prompts for pixel-level anomaly tracking, plus a dual pixel/object-level benchmark on UCSD Ped2 and ShanghaiTech.
Problem
Existing video anomaly detection methods are frame-centric or object-centric and lack pixel-level localization of anomalous objects, especially for overlapping or multiple anomalies.
Approach
Use object-centric VAD scores to select anomalous bounding boxes, filter them with a temporal tracking-based robustness algorithm, and feed the filtered boxes as prompts to SAM2 video tracking to produce pixel-level anomaly masks without fine-tuning SAM2.
Key results
- On UCSD Ped2, TAO reaches 83.6 RBDC and 93.2 TBDC, beating the best listed object-level baseline SiVL/AED-SSMTL (74 / 91.2).
- On ShanghaiTech Campus, TAO reports 62.1 RBDC and 85.4 TBDC versus STPT's 51.6 and 84.6.
- On UCSD Ped2 pixel metrics, TAO reaches 75.11 Pixel-AUROC and 64.12 Pixel-F1, well above fully trained DRAEM (69.58 and 10.89).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate object-centric anomaly detection, robust box filtering, and SAM2 video segmentation | TAO · UCSD Ped2 | Pixel-F1: 32.99 -> 64.12 | Helped |
| add video tracking mode (ablation) | TAO · UCSD Ped2 | Pixel-F1: 30.56 -> 64.12 | Helped |
| add box robust filtering (ablation) | TAO · UCSD Ped2 | Pixel-F1: 52.17 -> 64.12 | Helped |
| add video tracking mode and box robust filtering (ablation) | TAO · UCSD Ped2 | Pixel-F1: 32.99 -> 64.12 | Helped |
| add box robust filtering without video tracking (ablation) | TAO · UCSD Ped2 | Pixel-F1: 32.99 -> 30.56 | Hurt |
| replace raw thresholded boxes with robustly filtered boxes (ablation) | TAO · UCSD Ped2 | Pixel-F1: 52.17 -> 64.12 | Helped |
| vary the anomaly threshold τ (ablation) | TAO · ShanghaiTech Campus | TBDC | Mixed |
| vary SAM2 backbone from Tiny to Large (ablation) | TAO · UCSD Ped2 | pixel-level metrics | No clear effect |
Surprising findings
- Enabling box robust filtering alone yields minimal or no improvement in the ablation (Pixel-AUROC 41.72 to 39.06), while video tracking mode drives most of the gain.
Limitations
- Pixel-level ground truth is not available in existing VAD benchmarks, so the pixel-level evaluation relies on derived or proposed annotation and evaluation protocol.
- Performance depends on object-centric anomaly scoring thresholds (tau) and filtering hyperparameters tuned per dataset.
- Evaluated only on two datasets (UCSD Ped2 and ShanghaiTech Campus).
Experimental setup
Datasets: UCSD Ped2 ShanghaiTech Campus
Benchmarks: UCSD Ped2 ShanghaiTech Campus
Metrics: Pixel-AUROC Pixel-AP Pixel-AUPRO Pixel-F1 RBDC TBDC
Baselines: AdaCLIP AnomalyCLIP DRAEM DDAD SimpleNet OCAD SS-VAD SiVL AED-SSMTL HF2VAD BAF-AT STPT
Builds on: SAM2 SAM AED-SSMTL DRAEM
Closest papers by meaning
- Advancing Video Anomaly Detection: A Concise Review and a New Dataset (NeurIPS 2024 · similarity 0.89)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.89)
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.88)
- Open-Vocabulary Video Anomaly Detection (CVPR 2024 · similarity 0.88)
- MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection (CVPR 2024 · similarity 0.87)
- UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.