Research map / Label-efficient and robust learning / Knowledge distillation
Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition
Xiaobo Wang, Tianyu Fu, Shengcai Liao, Shuo Wang, Zhen Lei, Tao Mei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use feature consistency instead of probability consistency (ablation) | MobileFaceNet student · MegaFace identification | Rank-1 accuracy (%): 69.33 -> 83.96 | Improved |
| make feature consistency hardness-aware (HFC) (ablation) | feature consistency distillation · MegaFace identification | Rank-1 accuracy (%): 83.96 -> 84.19 | Improved |
| add softmax-based one-hot loss to feature consistency (ablation) | feature consistency distillation · MegaFace identification | Rank-1 accuracy (%): 83.96 -> 81.90 | Hurt |
| add softmax loss to probability consistency (ablation) | probability consistency distillation · MegaFace identification | Rank-1 accuracy (%): 69.33 -> 81.74 | Improved |
| add weight decay (5e-4 instead of 0) to feature consistency (ablation) | (1/2)MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 51.65 -> 74.32 | Improved |
| add weight exclusivity to feature consistency (ablation) | (1/2)MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 74.32 -> 75.29 | Improved |
| replace orthogonality regularization with weight exclusivity (ablation) | (1/2)MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 67.80 -> 75.29 | Improved |
| replace MHE with weight exclusivity (ablation) | (1/2)MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 74.58 -> 75.29 | Improved |
| double the number of filters (2x) versus original | MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 83.96 -> 87.19 | Improved |
| shrink filters to 1/8 of original | MobileFaceNet with FC · MegaFace identification | Rank-1 accuracy (%): 83.96 -> 25.32 | Hurt |
| use EC-KD instead of KD under 30% symmetric label noise | (1/2)MobileFaceNet student · MegaFace identification | Rank-1 accuracy (%): 25.53 -> 75.98 | Improved |
| train EC-KD on IQIYI-VID-Training instead of the student alone | (1/2)MobileFaceNet student · MegaFace identification (IQIYI-VID training) | Rank-1 accuracy (%): 29.62 -> 64.27 | Improved |
Closest papers by meaning
- Rethinking Feature-Based Knowledge Distillation for Face Recognition (CVPR 2023 · similarity 0.88)
- Grouped Knowledge Distillation for Deep Face Recognition (AAAI 2023 · similarity 0.88)
- AdaDistill: Adaptive Knowledge Distillation for Deep Face Recognition (ECCV 2024 · similarity 0.87)
- Evaluation-Oriented Knowledge Distillation for Deep Face Recognition (CVPR 2022 · similarity 0.87)
- Teacher Guided Neural Architecture Search for Face Recognition (AAAI 2021 · similarity 0.87)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
