Research map / Label-efficient and robust learning / Knowledge distillation
Student-Teacher Learning From Clean Inputs to Noisy Inputs
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Early-stop student-teacher training instead of training to zero loss (ablation) | Student-teacher learning · Noisy-input regression | test error | Helped |
| Train the student-teacher model to zero training loss rather than early-stop (ablation) | Student-teacher learning · Noisy-input regression | test error | Hurt |
| Use a more knowledgeable teacher, trained with more clean samples | Student-teacher learning · CIFAR10 | test error relative to base learning | Helped |
| Increase teacher training samples as student training samples increase | Student-teacher learning · CIFAR10 | test error relative to base learning | Mixed |
| Increase teacher training samples as input noise increases | Student-teacher learning · CIFAR10 | minimum teacher samples N_t: 200, 300, 500, 700, 800 | Helped |
| Pool more teacher neurons into each feature group (increase g) (ablation) | Student-teacher learning · Noisy-input regression with wide ReLU networks | test error | Hurt |
| Decompose teacher features into smaller groups (decrease g) (ablation) | Student-teacher learning · Noisy-input regression with wide ReLU networks | test error | Helped |
| Apply student-teacher learning when the student's sample amount is too small | Student-teacher learning · CIFAR10 | test error relative to base learning | No clear effect |
Closest papers by meaning
- How a Student becomes a Teacher: learning and forgetting through Spectral methods (NeurIPS 2023 · similarity 0.89)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.88)
- Improved Knowledge Distillation via Teacher Assistant (AAAI 2020 · similarity 0.88)
- Learning Student-Friendly Teacher Networks for Knowledge Distillation (NeurIPS 2021 · similarity 0.88)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.87)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.