Research map / Label-efficient and robust learning / Knowledge distillation
Weighted Distillation with Unlabeled Examples
Fotis Iliopoulos, Vasilis Kontonis, Cenk Baykal, Gaurav Menghani, Khoa Trinh, Erik Vee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reweight teacher-labeled examples with the proposed debiasing weights | Student distillation with unlabeled examples · CIFAR-10 | test accuracy | Improved |
| Reweight teacher-labeled examples with the proposed debiasing weights | Student distillation with unlabeled examples · CIFAR-100 | test accuracy | Improved |
| Reweight teacher-labeled examples with the proposed debiasing weights | Student distillation with unlabeled examples · SVHN | test accuracy | Improved |
| Reweight teacher-labeled examples with the proposed debiasing weights | Student distillation with unlabeled examples · CelebA gender classification | test accuracy | Improved |
| Reweight teacher-labeled examples with the proposed debiasing weights | Student distillation with unlabeled examples · ImageNet | test accuracy | Improved |
| Add the proposed example reweighting to VID | VID student distillation · CelebA gender classification | test accuracy | Improved |
| Update estimated weights at the end of every epoch instead of estimating them once (ablation) | Weighted student distillation · CIFAR-10 | test accuracy | Improved |
| Update estimated weights at the end of every epoch instead of estimating them once (ablation) | Weighted student distillation · CIFAR-100 | test accuracy | No clear effect |
| Update estimated weights at the end of every epoch instead of estimating them once (ablation) | Weighted student distillation · Other evaluated datasets | test accuracy | Improved |
| Apply the proposed weighting alongside temperature scaling (ablation) | Student distillation · CIFAR-100 | test accuracy | Improved |
| Use temperature 2.0 and apply the proposed weighting (ablation) | Student distillation · CIFAR-100 | test accuracy | Improved |
Closest papers by meaning
- Distilling Long-tailed Datasets (CVPR 2025 · similarity 0.85)
- Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks (ICLR 2025 · similarity 0.85)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.85)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.85)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.85)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
