Research map / Label-efficient and robust learning / Knowledge distillation
Co-Progression Knowledge Distillation with Knowledge Prototype for Industrial Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Co-Progression Knowledge Distillation with a Knowledge Prototype, letting the teacher and student co-evolve for unsupervised industrial anomaly detection on MVTec AD and VisA.
Problem
Knowledge-distillation anomaly detectors suffer from excessive specialization (student learns abnormal features) and inadequate generalization (student fails to match teacher on normal patterns) because teacher and student have similar structures.
Approach
Make the teacher bidirectionally learn from the student (co-progression) while regularizing it toward a coreset-based knowledge prototype built from a frozen teaching monitor model trained on normal samples, applicable to both forward (FD) and reverse (RD) distillation.
Key results
- CPKP(RD) reaches 99.55% average image-level AUROC and 98.01% pixel-level AUROC on MVTec AD, the reported SOTA.
- On VisA, CPKP(RD) achieves 96.99% image-level AUROC, second-best behind GLAD (99.46% vs. 96.99%? GLAD is listed at 99.46%), and 98.54% pixel-level AUROC.
- Ablation on MVTec: CP alone lowers FD/RD image AUROC (92.54%/96.41%), KP alone raises them (97.13%/98.97%), and both together give 98.39%/99.55%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Co-Progression and Knowledge Prototype to FD (ablation) | FD · MVTec AD | image-level AUC-ROC: 95.63 -> 98.39 | Helped |
| Add Co-Progression and Knowledge Prototype to RD (ablation) | RD · MVTec AD | image-level AUC-ROC: 98.46 -> 99.55 | Helped |
| Add Co-Progression without Knowledge Prototype to FD (ablation) | FD · MVTec AD | image-level AUC-ROC: 95.63 -> 92.54 | Hurt |
| Add Co-Progression without Knowledge Prototype to RD (ablation) | RD · MVTec AD | image-level AUC-ROC: 98.46 -> 96.41 | Hurt |
| Add Knowledge Prototype without Co-Progression to FD (ablation) | FD · MVTec AD | image-level AUC-ROC: 95.63 -> 97.13 | Helped |
| Add Knowledge Prototype without Co-Progression to RD (ablation) | RD · MVTec AD | image-level AUC-ROC: 98.46 -> 98.97 | Helped |
| Increase α from 0.9 to 0.999 (ablation) | CPKD(FD) · MVTec AD | image-level AUC-ROC: 97.61 -> 98.39 | Helped |
| Increase α from 0.9 to 0.999 (ablation) | CPKD(RD) · MVTec AD | image-level AUC-ROC: 98.51 -> 99.55 | Helped |
| Increase α from 0.99 to 0.999 (ablation) | CPKD(RD) · MVTec AD | pixel-level AUC-ROC: 98.10 -> 98.01 | Hurt |
| Use CPKP(RD) instead of RD | RD · VisA | image-level AUC-ROC: 95.93 -> 96.99 | Helped |
Surprising findings
- Co-progression without the knowledge prototype degrades performance below the baseline, suggesting the teacher can be corrupted by the student.
Limitations
- CP alone hurts performance, so the method depends on the knowledge prototype for stability.
- VisA results are not best overall; GLAD leads on image-level AUROC.
Experimental setup
Datasets: MVTec AD VisA
Benchmarks: MVTec AD VisA
Metrics: image-level AUROC pixel-level AUROC
Baselines: PatchCore SimpleNet GLAD FD RD RD++
Builds on: Reverse distillation STPM feature pyramid matching PatchCore WideResNet50
Closest papers by meaning
- Unlocking the Potential of Reverse Distillation for Anomaly Detection (AAAI 2025 · similarity 0.88)
- InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection (NeurIPS 2026 · similarity 0.86)
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.86)
- AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios (CVPR 2025 · similarity 0.86)
- Towards Training-free Anomaly Detection with Vision and Language Foundation Models (CVPR 2025 · similarity 0.86)
- Unsupervised Continual Anomaly Detection with Contrastively-Learned Prompt (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.