Research map / Label-efficient and robust learning / Knowledge distillation
Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again
Xin-Chun Li, Wen-shu Fan, Shaoming Song, Yinchuan Li, bingshuai Li, yunfeng shao, De-Chuan Zhan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply asymmetric temperatures (higher for correct class, lower for wrong classes) instead of uniform temperature in KD | KD with large teachers · CIFAR-100 | avg accuracy: 69.21 -> 70.94 | Improved |
| apply ATS | KD with large teachers · TinyImageNet/CUB/Dogs | avg accuracy: 58.89 -> 60.88 | Improved |
| ensemble two KD+ATS students | KD+ATS · CIFAR-100 | avg accuracy: 70.94 -> 73.16 | Improved |
| use a larger teacher with traditional KD instead of a smaller teacher | KD · CIFAR-100 | avg accuracy: 70.32 -> 69.21 | Hurt |
| add teacher assistant / early-stopped / customized teacher to KD with large teacher | KD · CIFAR-100 | avg accuracy: 69.21 -> 69.86 | Improved |
| remove class discriminability term (keep only correct guidance + smooth regularization, ILS) (ablation) | KD · CIFAR-100 | student accuracy | Hurt |
| set tau1 > tau2 in ATS rather than equal temperatures (ablation) | ATS · CIFAR-100, TinyImageNet | accuracy | Improved |
| add ATS across KD-loss weights lambda (ablation) | WRN28-8 -> MV2 · CIFAR-100 | accuracy | Improved |
Closest papers by meaning
- Knowledge Distillation Based on Transformed Teacher Matching (ICLR 2024 · similarity 0.88)
- All You Need in Knowledge Distillation Is a Tailored Coordinate System (AAAI 2025 · similarity 0.88)
- Logit Standardization in Knowledge Distillation (CVPR 2024 · similarity 0.87)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.87)
- Neural Collapse Inspired Knowledge Distillation (AAAI 2025 · similarity 0.86)
- Adaptive Dual Guidance Knowledge Distillation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
