Research map / Label-efficient and robust learning / Knowledge distillation
Probabilistic Knowledge Distillation of Face Ensembles
Jianqing Xu, Shen Li, Ailin Deng, Miao Xiong, Jiaying Wu, Jiaxiang Wu, Shouhong Ding, Bryan Hooi
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace mean averaging with Bayesian, confidence-weighted ensemble averaging | ResNet34 face ensemble · IJB-B | TAR at FAR 1e-5: 86.0 -> 89.7 | Improved |
| distill ensemble feature embeddings and BEA confidence statistic | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 vs 77.96 | Improved |
| distill ensemble feature embeddings and BEA confidence statistic | MobileFaceNet student · IJB-B | TAR at FAR 1e-5: 83.45 vs 81.85 | Improved |
| add Bayesian ensemble averaging rather than mean ensemble | ResNet34 face ensemble · CFP-FP | accuracy: 98.8 -> 99.0 | Improved |
| add Bayesian ensemble averaging rather than mean ensemble | ResNet34 face ensemble · CPLFW | accuracy: 93.7 -> 94.3 | Improved |
| distill from one deterministic teacher rather than BEA (ablation) | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 -> 77.91 | Hurt |
| distill from one SCF-trained teacher rather than BEA (ablation) | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 -> 80.55 | Hurt |
| distill from a deterministic teacher ensemble rather than BEA (ablation) | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 -> 78.92 | Hurt |
| distill from SCF-trained ensemble teachers rather than BEA (ablation) | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 -> 81.01 | Hurt |
| distill from one SCF teacher rather than BEA (ablation) | ResNet12 student · IJB-B | TAR at FAR 1e-5: 81.64 -> 80.93 | Hurt |
| add the BEA confidence statistic to weighted probabilistic ensemble distillation | face-recognition student models · IJB-B | TAR at FAR 1e-5 | Improved |
Closest papers by meaning
- Data Uncertainty Learning in Face Recognition (CVPR 2020 · similarity 0.86)
- Probabilistic Face Embeddings (ICCV 2019 · similarity 0.85)
- An Efficient Joint Formulation for Bayesian Face Verification (TPAMI 2016 · similarity 0.85)
- Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition (ECCV 2020 · similarity 0.85)
- Improving Training and Inference of Face Recognition Models via Random Temperature Scaling (AAAI 2023 · similarity 0.85)
- Bayesian Exploration of Pre-trained Models for Low-shot Image Classification (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
