Research map / Label-efficient and robust learning / Knowledge distillation
Face Model Compression by Distilling Knowledge from Neurons
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add neuron selection over top hidden-layer targets (ablation) | student mimicking T1 · LFW | AUC: 97.73 -> 98.18 | Helped |
| add neuron selection (ablation) | student mimicking T2 · LFW | AUC: 97.90 -> 98.12 | Helped |
| add neuron selection (ablation) | student mimicking T1 ensemble · LFW | AUC: 98.07 -> 98.25 | Helped |
| use hidden-layer neuron targets instead of soft targets (t=10) | student of T1 · LFW | AUC: 97.15 -> 97.73 | Helped |
| raise soft target temperature from 1 to 10 | student of T1 · LFW | AUC: 94.42 -> 97.15 | Helped |
| add attributes as extra supervision to 1-of-K student | S-1-of-K · LFW | accuracy: 91.23 -> 91% | No clear effect |
| add unsupervised fine-tuning on teacher features (ablation) | S-selection mimicking T1 ensemble · LFW | AUC: 98.25 -> 98.43 | Helped |
| compress T1 ensemble into small student | T1 ensemble · CPU inference | speed-up: 90x speed-up, 51.6x compression | Helped |
Closest papers by meaning
- Teacher Guided Neural Architecture Search for Face Recognition (AAAI 2021 · similarity 0.87)
- Masked Face Recognition with Generative-to-Discriminative Representations (ICML 2024 · similarity 0.87)
- Killing Two Birds With One Stone: Efficient and Robust Training of Face Recognition CNNs by Partial FC (CVPR 2022 · similarity 0.86)
- Sparsifying Neural Network Connections for Face Recognition (CVPR 2016 · similarity 0.86)
- Grouped Knowledge Distillation for Deep Face Recognition (AAAI 2023 · similarity 0.86)
- Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.