Research map / Label-efficient and robust learning / Knowledge distillation
Rethinking Feature-Based Knowledge Distillation for Face Recognition
Jingzhi Li, Zidong Guo, Hui Li, Seungju Han, Ji-won Baek, Min Yang, Ran Yang, Sungjoo Suh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add reverse distillation when training the teacher (ablation) | Feature-only distillation for face recognition · ICCV21-MFR MR-all | accuracy: 53.86 -> 57.27; 58.33 -> 63.04; 61.70 -> 66.13; 73.17 -> 75.07 | Improved |
| Add reverse distillation to tailor the teacher to IR18 (ablation) | IR100 → IR18 feature-only distillation · ICCV21-MFR MR-all | accuracy: 61.70 -> 66.13 | Improved |
| Use a half-depth student proxy in reverse distillation (ablation) | IR100 → IR18 feature-only distillation · ICCV21-MFR MR-all | TPR: 66.13 -> 68.56 | Improved |
| Use unlabeled data for student training (ablation) | ReFO+ · ICCV21-MFR MR-all | TPR: 68.56 -> 72.35; 59.17 -> 63.32 | Improved |
| Tailor IR100 to a student using reverse distillation (ablation) | IR100 teacher · ICCV21-MFR MR-all | accuracy: 79.06 -> 81.67 | Improved |
| Tailor the teacher to one student and use it for other students (ablation) | IR100 teacher and multiple student architectures · ICCV21-MFR MR-all | accuracy: 58.32 -> 63.22 | Improved |
| Reduce proxy depth from Sd=0.5 to Sd=0.25 (ablation) | MFN student proxy · ICCV21-MFR MR-all | accuracy | Hurt |
| Use depth reduction rather than channel slimming for the proxy (ablation) | MFN student proxy · ICCV21-MFR MR-all | accuracy | Improved |
| Change embedding normalization or the embedding distance loss (ablation) | IR100 → IR18 ReFO+ training · ICCV21-MFR MR-all | accuracy: 68.518; 68.500; 68.563; 68.239; 68.269 | No clear effect |
| Add ID supervision to the student loss (ablation) | IR100 → IR18 ReFO+ · ICCV21-MFR MR-all | accuracy | Hurt |
| Remove ID supervision without reverse distillation | Feature-based distillation · ICCV21-MFR MR-all | accuracy | Hurt |
| Use ReFO+ rather than the strongest ID-supervised baseline | IR100 → IR18 · ICCV21-MFR MR-all | TPR: 67.26 -> 68.56 | Improved |
Closest papers by meaning
- Grouped Knowledge Distillation for Deep Face Recognition (AAAI 2023 · similarity 0.91)
- ICD-Face: Intra-class Compactness Distillation for Face Recognition (ICCV 2023 · similarity 0.89)
- Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition (ECCV 2020 · similarity 0.88)
- Evaluation-Oriented Knowledge Distillation for Deep Face Recognition (CVPR 2022 · similarity 0.88)
- Improving Face Recognition from Hard Samples via Distribution Distillation Loss (ECCV 2020 · similarity 0.87)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
