Research map / Label-efficient and robust learning / Knowledge distillation
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Vasilis Kontonis, Fotis Iliopoulos, Khoa Trinh, Cenk Baykal, Gaurav Menghani, Erik Vee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add student-label mixing (SLaM) to vanilla distillation | soft distillation · CIFAR-10, 5000 labeled | accuracy: 63.53 -> 66.82 | Improved |
| add SLaM | soft distillation · CIFAR-100, 5000 labeled | accuracy: 37.94 -> 42.72 | Improved |
| add SLaM | soft distillation · ImageNet, 5% labeled | accuracy: 41.67 -> 48.1 | Improved |
| add SLaM | soft distillation · CelebA, 2% labeled | accuracy: 89.96 -> 90.37 | Improved |
| add SLaM | soft distillation · ImageNet, 30% labeled | accuracy: 69.12 -> 69.07 | No clear effect |
| increase labeled data from smallest to largest | SLaM gain over Vanilla · CIFAR-100 soft | accuracy: 42.72 -> 63.98 | Hurt |
| add SLaM | hard distillation · CIFAR-10, 5000 labeled | accuracy: 62.26 -> 66.40 | Improved |
| add SLaM | hard distillation · CIFAR-100, 5000 labeled | accuracy: 36.36 -> 42.01 | Improved |
| use hard instead of soft distillation | SLaM · CIFAR-10, 5000 labeled | accuracy: 66.82 -> 66.40 | Hurt |
| use hard instead of soft distillation | Vanilla · CIFAR-100, 5000 labeled | accuracy: 37.94 -> 36.36 | Hurt |
| add VID penultimate-layer matching | Vanilla · ImageNet, 5% labeled | accuracy: 41.67 -> 40.12 | Hurt |
| replace CE with Taylor-CE | Vanilla · CIFAR-100, 5000 labeled | accuracy: 37.94 -> 40.18 | Improved |
Closest papers by meaning
- The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works (ICML 2026 · similarity 0.89)
- Simple Unsupervised Knowledge Distillation With Space Similarity (ECCV 2024 · similarity 0.89)
- Robust Active Distillation (ICLR 2023 · similarity 0.89)
- Knowledge Distillation With the Reused Teacher Classifier (CVPR 2022 · similarity 0.88)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.88)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
