Research map / Vision-language and multimodal / CLIP and zero-shot
AA-CLIP: Enhancing Zero-Shot Anomaly Detection via Anomaly-Aware CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AA-CLIP, a two-stage residual-adapter adaptation of CLIP that first disentangles normal/anomaly text anchors and then aligns patch features to them, achieving zero-shot anomaly detection SOTA on 11 industrial and medical benchmarks.
Problem
CLIP is anomaly-unaware: normal and abnormal text embeddings overlap and patch features align poorly with anomaly semantics, limiting zero-shot anomaly detection and localization.
Approach
Adapt CLIP in two stages with lightweight residual adapters: first adapt the text encoder so normal and anomaly text anchors are orthogonal (disentangle loss), then adapt shallow visual layers so multi-granularity patch features align to these fixed anchors.
Key results
- Pixel-level average AUROC of 93.4% with full-shot training on VisA, a new SOTA over AnomalyCLIP (91.3%) and AdaCLIP (92.0%) in Table 1.
- Image-level average AUROC of 83.1% with 64-shot per class training, the best reported in Table 2.
- Ablation on VisA 64-shot: text-side residual adapter plus disentangle loss raises average AUROC from 88.9 (VAND-style linear projector) to 92.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AA-CLIP instead of AnomalyCLIP | Zero-shot anomaly detection · Industrial and Medical domains, average pixel-level | AUROC: 91.3 -> 93.4 | Helped |
| Use AA-CLIP instead of AnomalyCLIP | Zero-shot anomaly detection · Industrial and Medical domains, average image-level | AUROC: 80.6 -> 82.5 | Helped |
| Replace the linear adapter with a residual adapter (ablation) | CLIP visual feature adaptation · VisA-trained 64-shot setup | pixel-level AUROC: 48.9(-40.0) -> 91.3(+2.4) | Helped |
| Replace the linear adapter with a residual adapter (ablation) | CLIP visual feature adaptation · VisA-trained 64-shot setup | image-level AUROC: 53.4(-15.9) -> 80.7(+11.4) | Helped |
| Add text-space adaptation with residual adapters (ablation) | CLIP with visual residual adapters · VisA-trained 64-shot setup | pixel-level AUROC: 91.3(+2.4) -> 92.1(+3.2) | Helped |
| Add text-space adaptation with residual adapters (ablation) | CLIP with visual residual adapters · VisA-trained 64-shot setup | image-level AUROC: 80.7(+11.4) -> 82.6(+13.3) | Helped |
| Add the disentangle loss (ablation) | AA-CLIP with text and visual adaptation · VisA-trained 64-shot setup | pixel-level AUROC: 92.1 -> 92.7 | Helped |
| Add the disentangle loss (ablation) | AA-CLIP with text and visual adaptation · VisA-trained 64-shot setup | image-level AUROC: 82.6 -> 83.3 | Helped |
| Train text and visual encoders together in one stage instead of using two stages (ablation) | AA-CLIP · MVTec-AD | pixel-level AUROC: 90.1(-1.8) | Hurt |
| Train text and visual encoders together in one stage instead of using two stages (ablation) | AA-CLIP · MVTec-AD | image-level AUROC: 88.6(-1.9) | Hurt |
Surprising findings
- Original CLIP text embeddings for normal and anomaly prompts of the same class show high similarity, and single-stage joint adaptation of both encoders causes class information to collapse.
- Plain adapters without residual fusion severely degrade zero-shot performance (average AUROC drops to 48.9).
Limitations
- Signs of overfitting with full-shot training, suggesting saturation during CLIP adaptation.
- Image-level results are not uniformly best; e.g., MVTec-AD and some medical datasets trail other methods.
Experimental setup
Datasets: VisA MVTec AD BTAD MPDD BMAD CVC-ClinicDB CVC-ColonDB Kvasir-SEG CVC-300
Benchmarks: MVTec AD VisA BTAD MPDD BMAD (Brain MRI, Liver CT, Retina OCT) CVC-ClinicDB CVC-ColonDB Kvasir-SEG CVC-300
Metrics: AUROC (image-level) AUROC (pixel-level)
Baselines: CLIP WinCLIP VAND MVFA-AD AnomalyCLIP AdaCLIP
Builds on: CLIP VAND AnomalyCLIP WinCLIP
Closest papers by meaning
- AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection (ICLR 2024 · similarity 0.89)
- Aligning and Prompting Anything for Zero-Shot Generalized Anomaly Detection (AAAI 2025 · similarity 0.87)
- Unseen Visual Anomaly Generation (CVPR 2025 · similarity 0.86)
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.86)
- Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images (CVPR 2024 · similarity 0.86)
- Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.