Research map / Label-efficient and robust learning / OOD and anomaly detection
Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Scale-Aware Contrastive Reverse Distillation (SCRD4AD), which adds a noisy teacher fed simplex-noise-corrupted images for contrastive distillation and a scale adaptation module that softly weights per-scale losses, for unsupervised medical anomaly detection.
Problem
Unsupervised anomaly detection in medical images, where reverse distillation models have limited feature discriminability and cannot handle anomalies of varying scale.
Approach
Adds a weight-shared "noisy" teacher that encodes simplex-noise-synthesized anomalies so the student is pushed away from out-of-normal features while staying close to the clean teacher, plus an input-conditioned scale descriptor that reweights distillation losses across layers.
Key results
- On RSNA, the method reaches 91.01 AUC, versus 88.00 for the second-best method (RD++).
- On Brain Tumor, it reaches 98.88 AUC (4.38-point gain over the second-best method); on ISIC it reaches 83.10 AUC (3.08-point gain over ReContrast).
- Ablation on RSNA: contrastive reverse distillation raises AUC from 84.29 to 89.87; adding scale adaptation raises it to 91.01.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive reverse distillation to RD4AD (ablation) | SCRD · RSNA | AUC: 84.29 -> 89.87 | Helped |
| add contrastive reverse distillation to RD4AD (ablation) | SCRD · Brain Tumor | AUC: 90.52 -> 91.45 | Helped |
| add contrastive reverse distillation to RD4AD (ablation) | SCRD · ISIC | AUC: 76.48 -> 78.87 | Helped |
| add scale-adaptive mechanism to CRD (ablation) | SCRD · RSNA | AUC: 89.87 -> 91.01 | Helped |
| add scale-adaptive mechanism to CRD (ablation) | SCRD · Brain Tumor | AUC: 91.45 -> 98.88 | Helped |
| add scale-adaptive mechanism to CRD (ablation) | SCRD · ISIC | AUC: 78.87 -> 83.10 | Helped |
| replace simplex noise with Gaussian noise for synthetic anomalies (ablation) | SCRD · RSNA | AUC: 90.70 vs 91.01 | Mixed |
| replace simplex noise with Gaussian noise for synthetic anomalies (ablation) | SCRD · Brain Tumor | AUC: 75.23 vs 98.88 | Hurt |
| replace simplex noise with Gaussian noise for synthetic anomalies (ablation) | SCRD · ISIC | AUC: 74.30 vs 83.10 | Hurt |
| increase simplex-noise intensity from moderate to excessive (ablation) | SCRD on RSNA · RSNA | AUC | Hurt |
Surprising findings
- Scale adaptation gives only a 1.14-point AUC gain on RSNA, versus 7.43 points on Brain Tumor, which the authors attribute to more uniform anomaly sizes in RSNA.
Limitations
- Synthetic anomalies rely on random spatial sampling of simplex noise, which the authors note is a limitation.
- Reported results on ISIC and Brain Tumor are noticeably lower or variable across methods, and the authors attribute ISIC difficulty to subtle lesion morphology.
Experimental setup
Datasets: RSNA Pneumonia Detection Challenge Brain Tumor MRI dataset ISIC 2018
Benchmarks: RSNA Brain Tumor MRI ISIC 2018
Metrics: AUC F1 accuracy
Baselines: AE UAE MorphAEus GAN Ensemble MemAE PatchCore SQUID FastFlow CFLOW-AD CutPaste NSA RD4AD
Builds on: RD4AD RD++ ReContrast CutPaste
Closest papers by meaning
- Unlocking the Potential of Reverse Distillation for Anomaly Detection (AAAI 2025 · similarity 0.88)
- Generating and Reweighting Dense Contrastive Patterns for Unsupervised Anomaly Detection (AAAI 2024 · similarity 0.85)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.85)
- Boosting Fine-Grained Visual Anomaly Detection with Coarse-Knowledge-Aware Adversarial Learning (AAAI 2025 · similarity 0.84)
- Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration (AAAI 2025 · similarity 0.84)
- Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.