AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Rethinking Feature-Based Knowledge Distillation for Face Recognition

Jingzhi Li, Zidong Guo, Hui Li, Seungju Han, Ji-won Baek, Min Yang, Ran Yang, Sungjoo Suh

CVPR 2023 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 48 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add reverse distillation when training the teacher (ablation)Feature-only distillation for face recognition · ICCV21-MFR MR-allaccuracy: 53.86 -> 57.27; 58.33 -> 63.04; 61.70 -> 66.13; 73.17 -> 75.07Improved
Add reverse distillation to tailor the teacher to IR18 (ablation)IR100 → IR18 feature-only distillation · ICCV21-MFR MR-allaccuracy: 61.70 -> 66.13Improved
Use a half-depth student proxy in reverse distillation (ablation)IR100 → IR18 feature-only distillation · ICCV21-MFR MR-allTPR: 66.13 -> 68.56Improved
Use unlabeled data for student training (ablation)ReFO+ · ICCV21-MFR MR-allTPR: 68.56 -> 72.35; 59.17 -> 63.32Improved
Tailor IR100 to a student using reverse distillation (ablation)IR100 teacher · ICCV21-MFR MR-allaccuracy: 79.06 -> 81.67Improved
Tailor the teacher to one student and use it for other students (ablation)IR100 teacher and multiple student architectures · ICCV21-MFR MR-allaccuracy: 58.32 -> 63.22Improved
Reduce proxy depth from Sd=0.5 to Sd=0.25 (ablation)MFN student proxy · ICCV21-MFR MR-allaccuracyHurt
Use depth reduction rather than channel slimming for the proxy (ablation)MFN student proxy · ICCV21-MFR MR-allaccuracyImproved
Change embedding normalization or the embedding distance loss (ablation)IR100 → IR18 ReFO+ training · ICCV21-MFR MR-allaccuracy: 68.518; 68.500; 68.563; 68.239; 68.269No clear effect
Add ID supervision to the student loss (ablation)IR100 → IR18 ReFO+ · ICCV21-MFR MR-allaccuracyHurt
Remove ID supervision without reverse distillationFeature-based distillation · ICCV21-MFR MR-allaccuracyHurt
Use ReFO+ rather than the strongest ID-supervised baselineIR100 → IR18 · ICCV21-MFR MR-allTPR: 67.26 -> 68.56Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy