Research map / Vision-language and multimodal / Human and embodied modeling
Towards Training-free Anomaly Detection with Vision and Language Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LogSAD, a training-free framework that uses GPT-4V-generated matching rules plus patch, interest-set, and CLIP-based composition detectors to detect both logical and structural anomalies on MVTec LOCO, MVTec AD and VisA.
Problem
Existing anomaly detectors mainly catch local structural defects and struggle with logical/compositional anomalies, and methods that handle both typically need training and mask annotations.
Approach
Match-of-thought uses GPT-4V to turn logical constraints and few normal images into interests of thought and compositional matching rules; three granularity detectors (patch memory bank, bipartite interest matching with CLIP+SAM segmentation, and CLIP zero-shot composition checks) are calibrated, standardized, and fused by max, with no training.
Key results
- On MVTec LOCO full-data, LogSAD averages 90.2 image-level AUROC versus 87.5 for GRAD and 86.0 for THFR, without training.
- In 4-shot MVTec LOCO, LogSAD reaches 86.3 average AUROC, improving 17.4% over PromptAD (reported in the paper).
- Mean logical/structural AUROC on MVTec LOCO is 89.3/93.1, versus EfficientAD's 86.8/94.7 and PSAD's 98.1/91.6 (PSAD excluded as it uses extra annotations).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LogSAD instead of GRAD | MVTec LOCO full-data protocol · MVTec LOCO | image-level AUROC: 87.5 -> 90.2 | Helped |
| Use LogSAD instead of PromptAD | MVTec LOCO few-shot protocols · MVTec LOCO | image-level AUROC: 72.4 -> 82.3 | Helped |
| Use LogSAD instead of EfficientAD | MVTec LOCO logical anomaly detection · MVTec LOCO, logical anomalies | AUROC: 86.8 -> 89.3 | Helped |
| Use LogSAD instead of EfficientAD | MVTec LOCO structural anomaly detection · MVTec LOCO, structural anomalies | AUROC: 94.7 -> 93.1 | Hurt |
| Use LogSAD instead of PromptAD | MVTec AD 4-shot · MVTec AD | image-level AUROC: 96.6 -> 97.0 | Helped |
| Use LogSAD instead of PromptAD | VisA 4-shot · VisA | image-level AUROC: 89.1 -> 93.0 | Helped |
| Add patch matching (ablation) | LogSAD · MVTec LOCO, structural anomalies | image-level AUROC | Helped |
| Add interest matching (ablation) | LogSAD · MVTec LOCO, structural and logical anomalies | image-level AUROC | Helped |
| Add composition matching (ablation) | LogSAD · MVTec LOCO, logical anomalies | image-level AUROC | Helped |
| Add composition matching to patch and interest matching (ablation) | LogSAD · MVTec LOCO | image-level AUROC: 85.1 -> 90.2 | Helped |
| Add composition matching to patch and interest matching (ablation) | LogSAD · MVTec LOCO | image-level F1-max: 83.8 -> 88.8 | Helped |
| Add interest matching to patch and composition matching (ablation) | LogSAD · MVTec LOCO | image-level AUROC: 86.4 -> 90.2 | Helped |
Surprising findings
- Off-the-shelf LMMs (LLaVA, Mini-Gemini) hallucinate on counting and left/right clamp questions, which motivated replacing them with a CLIP+SAM composition detector.
Limitations
- Open-vocabulary segmentation quality limits interest matching and composition checks.
- Relies on GPT-4V and foundation models with reference normal images and manually curated rules per category.
- Logical AUROC (89.3) lags PSAD, which uses extra segment annotations.
Experimental setup
Datasets: MVTec LOCO MVTec AD VisA
Benchmarks: MVTec LOCO MVTec AD VisA
Metrics: image-level AUROC pixel-level AUROC image-level F1-max
Baselines: SPADE PaDim GCAD PatchCore THFR SINBAD GRAD GeneralAD WinCLIP+ PromptAD AnomalyGPT DSR
Builds on: PatchCore chain-of-thought WinCLIP SPADE
Closest papers by meaning
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.88)
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.88)
- LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · similarity 0.88)
- Supervised Anomaly Detection for Complex Industrial Images (CVPR 2024 · similarity 0.87)
- AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios (CVPR 2025 · similarity 0.87)
- AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.