Research map / Vision-language and multimodal / CLIP and zero-shot
Aligning and Prompting Anything for Zero-Shot Generalized Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TPS, which generates fine-grained "shunt" text prompts for pixel-level anomaly segmentation alongside global prompts for classification, linked by learned pathway modules on frozen CLIP, for zero-shot generalized anomaly detection.
Problem
Zero-shot generalized anomaly detection with CLIP uses one set of broad text prompts for both image-level classification and pixel-level segmentation, which have different granularity, leading to poor visual-text alignment for segmentation.
Approach
Split text prompts into global prompts for classification and fine-grained shunt prompts (flawless/damaged part) for segmentation, and connect them with cross-attention pathway modules that use multi-level CLIP patch embeddings, trained jointly with CE, focal, and dice losses while CLIP stays frozen.
Key results
- TPS reaches 96.4 AUROC / 98.4 AP image-level classification on MVTec AD, versus 97.8/99.1 for MuSc (MuSc is higher on that dataset).
- TPS achieves the best pixel-level AUPRO on several datasets, e.g., 93.3 AUROC / 71.2 AP on MSD vs. 88.5/60.8 for WinCLIP, and 91.2 AUROC on ISIC vs. 89.7 for AnomalyCLIP.
- Training TPS on an auxiliary medical dataset (HeadCT for classification, ISIC for segmentation) raises HeadCT classification AUROC from 92.3 to 99.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TPS instead of the strongest listed baseline | Zero-shot generalized anomaly classification · MVTec AD, VisA, MPDD, BTAD, HeadCT, BrainMRI, Br35H | AUROC | Mixed |
| Use TPS instead of the strongest listed baseline | Zero-shot generalized anomaly segmentation · MVTec AD, VisA, MPDD, BTAD, SD-saliency-900, RSDDS-113, ISIC, Kvasir, TN3k, MSD | AUROC | Mixed |
| Add fine-grained shunt text prompts for segmentation and improved global text prompts for classification (ablation) | TPS · ZGAD benchmarks in Figure 3 | AUROC | Helped |
| Add the pathway module to connect classification and segmentation (ablation) | TPS · ZGAD benchmarks in Figure 3 | AUROC | Helped |
Surprising findings
- MuSc, which uses no text prompts, outperforms TPS on several industrial classification and segmentation benchmarks but performs poorly on medical images.
Limitations
- Performance on medical datasets is weaker than on industrial datasets without medical auxiliary training.
- Requires an auxiliary training dataset (MVTec AD test set plus VisA) and dataset-specific auxiliary choices.
Experimental setup
Datasets: MVTec AD VisA MPDD BTAD SD-saliency-900 RSDDS-113 HeadCT BrainMRI Br35H ISIC Kvasir TN3k
Benchmarks: MVTec AD VisA MPDD BTAD SD-saliency-900 RSDDS-113 HeadCT BrainMRI Br35H ISIC Kvasir TN3k
Metrics: AUROC AP AUPRO
Baselines: CLIP WinCLIP AnomalyCLIP MuSc
Builds on: CLIP WinCLIP AnomalyCLIP
Closest papers by meaning
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection (ICLR 2024 · similarity 0.90)
- Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection (CVPR 2025 · similarity 0.89)
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.88)
- AA-CLIP: Enhancing Zero-Shot Anomaly Detection via Anomaly-Aware CLIP (CVPR 2025 · similarity 0.87)
- DZAD: Diffusion-based Zero-shot Anomaly Detection (AAAI 2025 · similarity 0.87)
- Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.