Research map / Label-efficient and robust learning / Noisy labels
Understanding Self-Distillation in the Presence of Label Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the imitation weight beyond 1 (ablation) | Self-distillation on Caltech-256 with ResNet-34 · 50% random corruption | improvement over teacher (percentage points): 8.54 ± 0.29 -> 10.04 ± 0.51 | Helped |
| Increase the imitation weight beyond 1 (ablation) | Self-distillation on Caltech-256 with VGG-16 · 50% random corruption | improvement over teacher (percentage points): 2.01 ± 0.14 -> 5.86 ± 0.18 | Helped |
| Increase the imitation weight from 1 to 1.2 (ablation) | Self-distillation on CIFAR-100 with ResNet-34 · 50% hierarchical corruption | improvement over teacher (percentage points): 4.19 ± 0.09 -> 4.46 ± 0.19 | Helped |
| Increase the imitation weight beyond 1 (ablation) | Self-distillation on CIFAR-100 with VGG-16 · 50% hierarchical corruption | improvement over teacher (percentage points): 5.29 ± 0.11 -> 7.23 ± 0.17 | Helped |
| Increase the imitation weight beyond 1 (ablation) | Self-distillation on Food-101 with VGG-16 · 50% adversarial corruption | improvement over teacher (percentage points): 3.85 ± 0.05 -> 4.39 ± 0.29 | Helped |
| Increase the imitation weight from 1 to 1.2 (ablation) | Self-distillation on Food-101 with ResNet-34 · 50% adversarial corruption | improvement over teacher (percentage points): 1.85 ± 0.09 -> 1.87 ± 0.06 | No clear effect |
| Train with teacher predictions only (ξ=1) | Self-distillation on Caltech-256 with ResNet-34 · 0% random corruption | improvement over teacher (percentage points): -0.04 ± 0.02% | Hurt |
| Train with teacher predictions only (ξ=1) | Self-distillation on CIFAR-100 with ResNet-34 · 0% hierarchical corruption | improvement over teacher (percentage points): -0.23 ± 0.06% | Hurt |
| Train with teacher predictions only (ξ=1) | Self-distillation on Food-101 with ResNet-34 · 0% adversarial corruption | improvement over teacher (percentage points): -0.37 ± 0.10% | Hurt |
| Increase label corruption from 0% to 50% | Self-distillation with ξ=1, ResNet-34 · Caltech-256, random corruption | improvement over teacher (percentage points): -0.04 ± 0.02% -> 8.54 ± 0.29% | Helped |
| Increase label corruption from 0% to 50% | Self-distillation with ξ=1, ResNet-34 · CIFAR-100, hierarchical corruption | improvement over teacher (percentage points): -0.23 ± 0.06% -> 4.19 ± 0.09% | Helped |
| Increase label corruption from 0% to 50% | Self-distillation with ξ=1, ResNet-34 · Food-101, adversarial corruption | improvement over teacher (percentage points): -0.37 ± 0.10% -> 1.85 ± 0.09% | Helped |
Closest papers by meaning
- The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model (NeurIPS 2025 · similarity 0.91)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.89)
- Self-cognitive Denoising in the Presence of Multiple Noisy Label Sources (ICML 2024 · similarity 0.89)
- Rethinking Self-Distillation: Label Averaging and Enhanced Soft Label Refinement with Partial Labels (ICLR 2025 · similarity 0.88)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.86)
- GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.