Research map / Label-efficient and robust learning / OOD and anomaly detection
ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ADPretrain, a contrastive pretraining framework that learns anomaly-discriminative representations on residual features from the RealIAD dataset, using angle- and norm-oriented contrastive losses, and plugs them into embedding-based AD methods.
Problem
ImageNet-pretrained features are suboptimal for industrial anomaly detection because pretraining does not target normal-vs-abnormal discrimination and natural images differ in distribution from industrial AD images.
Approach
Learn a frozen-backbone Feature Projector (learnable key/value attention Transformer) on residual features from RealIAD, optimized with an angle-oriented InfoNCE variant (center-shifted cosine, normal-vs-abnormal negatives only) and a norm-oriented hypersphere contraction/push loss, so the resulting features can replace ImageNet features in existing AD methods.
Key results
- Replacing original features with ADPretrain features improves PatchCore on MVTecAD with DINOv2-Base from 95.5/82.7 to 99.0/87.4 (image AUROC/PRO).
- Across five embedding-based AD methods and five backbones, the pretrained features consistently improve results on most datasets, e.g., FeatureNorm with ImageBind reaches 94.2 image AUROC on MPDD vs 44.9 original.
- In 2-shot FSAD on VisA, FeatureNorm (ours) achieves 92.4 I-AUROC and 87.5 PRO, outperforming ResAD and most listed FSAD methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace PatchCore's original features with ADPretrain features | PatchCore with ImageBind · MVTecAD | image AUROC: 98.5 -> 98.9 (+0.4) | Helped |
| Replace PatchCore's original features with ADPretrain features | PatchCore with ImageBind · VisA | image AUROC: 91.4 -> 94.8 (+3.4) | Helped |
| Replace PatchCore's original features with ADPretrain features | PatchCore with ImageBind · BTAD | image AUROC: 94.6 -> 95.6 (+1.0) | Helped |
| Replace PatchCore's original features with ADPretrain features | PatchCore with ImageBind · MVTec3D | image AUROC: 78.4 -> 82.6 (+4.2) | Helped |
| Replace PatchCore's original features with ADPretrain features | PatchCore with ImageBind · MPDD | image AUROC: 92.6 -> 94.8 (+2.2) | Helped |
| Use residual features instead of vanilla backbone features (ablation) | ADPretrain with ImageBind · VisA | PaDiM image AUROC: 93.5 -> 95.4 | Helped |
| Combine angle and norm contrastive losses instead of using either alone (ablation) | ADPretrain with ImageBind · VisA | PaDiM image AUROC: 93.9 -> 95.4; 93.7 -> 95.4 | Helped |
| Keep the backbone fixed instead of training it (ablation) | ADPretrain with ImageBind · VisA | PaDiM image AUROC: 89.3 -> 93.5; 81.3 -> 95.4 | Helped |
| Use Learnable Key/Value Attention instead of self-attention (ablation) | ADPretrain Feature Projector · VisA | PaDiM image AUROC: 93.7 -> 95.5 | Helped |
| Use ADPretrain feature norms as anomaly scores | FeatureNorm with ImageBind · VisA | image AUROC: 52.2 -> 95.3 | Helped |
| Pretrain on RealIAD instead of MVTecAD (ablation) | ADPretrain · VisA | PaDiM image AUROC: 93.0 -> 95.6 | Helped |
| Use ADPretrain features with only 10% normal downstream training samples | PaDiM · MPDD | image AUROC: 63.1 -> 91.4 | Helped |
Surprising findings
- Pretraining on MVTecAD itself (data leakage setting) did not give better gains than pretraining on RealIAD, suggesting leakage is not the main driver.
- Fine-tuning the full backbone during pretraining degraded performance, so the backbone is kept frozen.
Limitations
- Backbone is frozen; only the Feature Projector is trained, so it is not true full-network pretraining.
- Only applicable to embedding-based AD methods relying on pretrained feature extractors; not diffusion-based methods.
- Few-shot reference set sensitivity: non-representative reference sets degrade residual features.
Experimental setup
Datasets: RealIAD MVTecAD VisA BTAD MVTec3D MPDD
Benchmarks: MVTecAD VisA BTAD MVTec3D MPDD
Metrics: image-level AUROC PRO pixel-level AUROC
Baselines: PaDiM PatchCore CFLOW GLASS UniAD FYD PANDA MSC WinCLIP KAG-Prompt ResAD PromptAD
Builds on: ResAD InfoNCE PatchCore PaDiM GLASS
Closest papers by meaning
- ResAD: A Simple Framework for Class Generalizable Anomaly Detection (NeurIPS 2024 · similarity 0.89)
- AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios (CVPR 2025 · similarity 0.89)
- Long-Tailed Anomaly Detection with Learnable Class Names (CVPR 2024 · similarity 0.87)
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.87)
- An Evidence-Based Post-Hoc Adjustment Framework for Anomaly Detection Under Data Contamination (NeurIPS 2025 · similarity 0.87)
- AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.