Research map / Label-efficient and robust learning / Knowledge distillation

Unlocking the Potential of Reverse Distillation for Anomaly Detection

AAAI 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Distillation with causal and adaptive guidance · 17 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reverse Distillation with Expert (RD-E), an Expert-Teacher-Student network with synthetic-anomaly-guided distillation, plus Guided Information Injection (GII) for skip-like detail transfer, improving unsupervised anomaly detection under the RD paradigm.

Problem

Reverse Distillation for unsupervised anomaly detection suffers from missed detections (teacher and student insufficiently differ in anomalous regions due to student over-generalization) and false positives (lack of skip connections loses fine detail in reconstruction).

Approach

Add a frozen-at-inference expert encoder (trained only on normal images) that jointly distills the teacher (anomaly sensitivity via synthetic anomaly masks) and the student (denoising), and inject teacher detail features into the decoder via similarity-attention-weighted GII instead of raw skip connections.

Key results

• On MVTec AD, pixel-level P-AUC of 99.0% exceeds prior KD-based SOTA, and the method ranks best among RD-based methods in P-AP (72.4%) and P-PRO (96.3%).
• Image-level AP on MVTec AD reaches 99.7%, state of the art, while I-AUC (99.2%) is comparable to MemKD (99.6%).
• Ablation on MVTec AD shows RD-E raises P-AP from 61.01 to 71.80 and GII with similarity attention brings P-AP to 72.37.

Measured effects reported in the paper

Surprising findings

• On MPDD, the method does not beat MemKD in image-level detection, and on BTAD class 02 image-level AUC is lower than RD++ (85.8 vs 87.3).
• Naive skip connections in GII improve localization less than the similarity-attention variant but still help, though the paper argues raw skip connections risk anomaly leakage.

Limitations

• Image-level AUC on MVTec AD is not the best; MemKD leads in I-AUC (99.6 vs 99.2).
• Requires synthetic anomaly generation and ground-truth anomaly masks for teacher distillation, and an extra expert network during training.
• Results on MPDD image-level detection fall short of MemKD.

Experimental setup

Datasets: MVTec AD MPDD BTAD VisA DTD ImageNet

Benchmarks: MVTec AD MPDD BTAD VisA

Metrics: I-AUC I-AP P-AUC P-AP P-PRO

Baselines: STPM DeSTSeg HypAD RD RD++ THFR MemKD

Builds on: Reverse Distillation DRÆM ReContrast DeSTSeg

Closest papers by meaning

• Co-Progression Knowledge Distillation with Knowledge Prototype for Industrial Anomaly Detection (AAAI 2025 · similarity 0.88)
• Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection (ICLR 2025 · similarity 0.88)
• ResAD: A Simple Framework for Class Generalizable Anomaly Detection (NeurIPS 2024 · similarity 0.87)
• Rethinking Reverse Distillation for Multi-Modal Anomaly Detection (AAAI 2024 · similarity 0.87)
• CNC: Cross-modal Normality Constraint for Unsupervised Multi-class Anomaly Detection (AAAI 2025 · similarity 0.87)
• AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios (CVPR 2025 · similarity 0.86)