Research map / Label-efficient and robust learning / Knowledge distillation
Learning Student Networks with Few Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add WaGe loss on top of KD | KD · CIFAR-10 1000 per class | accuracy: 77.63% -> 82.01% | Helped |
| add WaGe loss on top of KD | KD · CIFAR-10 500 per class | accuracy: 50.63% -> 73.08% | Helped |
| add WaGe loss on top of KD | KD · CIFAR-10 full data | accuracy: 79.72% -> 84.66% | Helped |
| add WaGe loss on top of KD | KD · Fashion-MNIST 50 per class | accuracy: 78.5% -> 81.73% | Helped |
| add WaGe loss on top of KD | KD · CIFAR-100 250 per class | accuracy: 34.7% -> 34.98% | Mixed |
| WaGe vs Activation Boundary loss | KD+WaGe · CIFAR-10 Student #2 1000 per class | accuracy: 78.64% -> 84.23% | Helped |
| increase WaGe weight alpha beyond 0.001 (ablation) | KD+WaGe · CIFAR-10 M=1000 | accuracy: 80.65% -> 63.16% | Hurt |
| vary Wasserstein radius epsilon at fixed alpha (ablation) | KD+WaGe · CIFAR-10 M=1000 | accuracy | No clear effect |
Closest papers by meaning
- Learning Student Networks in the Wild (CVPR 2021 · similarity 0.89)
- How a Student becomes a Teacher: learning and forgetting through Spectral methods (NeurIPS 2023 · similarity 0.88)
- Adversarial Learning of Portable Student Networks (AAAI 2018 · similarity 0.88)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.86)
- Weak-to-Strong Generalization Even in Random Feature Networks, Provably (ICML 2025 · similarity 0.85)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.