Research map / Label-efficient and robust learning / Knowledge distillation
Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition
Sungho Shin, Joosoon Lee, Junseok Lee, Yeonguk Yu, Kyoobin Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add A-SKD attention distillation | LR ResNet student · AgeDB-30 8x | verification accuracy: 77.75 -> 79.00 | Improved |
| add A-SKD attention distillation | LR ResNet student · AgeDB-30 4x | verification accuracy: 87.74 -> 88.58 | Improved |
| add logit distillation to A-SKD (ablation) | A-SKD · AgeDB-30 4x | verification accuracy: 88.58 -> 89.15 | Improved |
| use feature L2 distillation (F-KD) | LR student · AgeDB-30 4x | verification accuracy: 87.74 -> 87.72 | No clear effect |
| use attention transfer AT with L2 loss | LR student · TinyFace | rank-1 accuracy: 42.19 -> 36.56 | Hurt |
| use A-SKD | LR student · TinyFace | rank-1 accuracy: 42.19 -> 47.91 | Improved |
| replace L2 with cosine loss in attention transfer (ablation) | AT · AgeDB-30 8x | verification accuracy: 77.40 -> 77.72 | Improved |
| replace non-parametric with parametric attention layers (ablation) | AT* · AgeDB-30 8x | verification accuracy: 77.72 -> 78.66 | Improved |
| add channel attention to spatial attention (ablation) | A-SKD · AgeDB-30 8x | verification accuracy: 78.66 -> 79.00 | Improved |
| apply A-SKD on ImageNet classification | ResNet18-CBAM · ImageNet 4x | accuracy: 65.34 -> 66.52 | Improved |
| apply A-SKD on face detection | TinaFace-CBAM · WIDER FACE Hard 16x | mAP: 35.29 -> 47.28 | Improved |
| apply A-SKD on face detection | TinaFace-CBAM · WIDER FACE Easy 32x | mAP: 31.15 -> 33.50 | Improved |
Closest papers by meaning
- Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition (AAAI 2020 · similarity 0.90)
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (TPAMI 2021 · similarity 0.89)
- NORM: Knowledge Distillation via N-to-One Representation Matching (ICLR 2023 · similarity 0.88)
- Teacher Guided Neural Architecture Search for Face Recognition (AAAI 2021 · similarity 0.88)
- No Head Left Behind – Multi-Head Alignment Distillation for Transformers (AAAI 2024 · similarity 0.87)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
