Research map / Video / Video-language understanding
Towards Visual Discrimination and Reasoning of Real-World Physical Dynamics: Physics-Grounded Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Phys-AD, a real-robot video dataset (6,400+ videos, 22 categories, 47 anomaly types) for physics-grounded industrial anomaly detection, benchmarks unsupervised, weakly-supervised, and video-language methods, and proposes the PAEval metric for explaining anomalies.
Problem
Existing industrial anomaly detection datasets are static, single-view, and semantically simple, lacking physical priors, object interaction, and temporal reasoning needed for real-world physics-grounded anomalies.
Approach
A large-scale real-world video dataset where objects are physically interacted with by robotic arms and motors, paired with a benchmark across three AD settings and a PAEval metric that scores VLM classification, description, and physical explanation of anomalies.
Key results
- Best average video-level AUROC is only 66.9% (MNAD.r on per-category average 0.669 reported), and the best VLM (Video-LLaMA) reaches 52.3% average AUROC.
- On PAEval, the best VLM achieves only 21.9% description and 30.3% explanation scores.
- Unsupervised MNAD.p reaches 81.0% AUROC on fan and 68.0% on screw, showing some strength on temporal anomalies.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MNAD.r instead of the other evaluated methods | Video anomaly detection on Phys-AD · Phys-AD | AUROC: 0.669 | Helped |
| Use MNAD.p for temporal anomaly detection | Unsupervised anomaly detection · Fan; screw | AUROC: 81.0% for fan, 68.0% for screw | Helped |
| Use weakly supervised training with anomaly samples | Weakly-supervised anomaly detection · Phys-AD | AUROC | Mixed |
| Use MemAE for spatial anomaly detection | Unsupervised anomaly detection · Sticky Roller | AUROC: 96.7% | Helped |
| Use MemAE for temporal anomalies rather than spatial anomalies | Unsupervised anomaly detection · Rolling Bearing; Spherical Bearing | AUROC: 4.4% and 9.2% | Hurt |
| Use MNAD with temporal prototypes | Unsupervised anomaly detection · Bearing anomaly classes in Phys-AD | AUROC | Helped |
| Use pretrained video-language models for physics-grounded anomaly detection | Video-language models · Phys-AD | AUROC: 52.3% | Hurt |
| Use Video-LLaVA for descriptions instead of the other evaluated VLMs | Visual-language models · Phys-AD | Description: 21.9% | Helped |
| Use Video-LLaMA for physical explanations instead of the other evaluated VLMs | Visual-language models · Phys-AD | Explanation: 30.3% | Helped |
Surprising findings
- Video-language models perform near chance (about 50-52% average AUROC), and several zero-shot CLIP-based methods score exactly 0.500 across categories.
- Unsupervised MemAE scores 4.4% AUROC on rolling bearing, well below chance.
Limitations
- Most existing anomaly detection algorithms cannot be directly applied to the dataset.
- Limited to a fixed set of objects and interaction methods; more diversity planned.
- Zero-shot, few-shot, and semi-supervised settings not yet evaluated.
Experimental setup
Datasets: Phys-AD
Benchmarks: Phys-AD
Metrics: AUROC AP accuracy PAEval classification PAEval description PAEval explanation
Baselines: MemAE MNAD MPN VadCLIP MGFN S3R LAVAD Video-ChatGPT Video-LLaMA Video-LLaVA CLIP ImageBind
Builds on: MVTec-AD PAEval
Closest papers by meaning
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.87)
- Multi-Sensor Object Anomaly Detection: Unifying Appearance, Geometry, and Internal Properties (CVPR 2025 · similarity 0.86)
- ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining (NeurIPS 2025 · similarity 0.85)
- Real-IAD D3: A Real-World 2D/Pseudo-3D/3D Dataset for Industrial Anomaly Detection (CVPR 2025 · similarity 0.84)
- PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer (NeurIPS 2025 · similarity 0.84)
- Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.