Research map / Label-efficient and robust learning / Knowledge distillation
Learning to Teach with Dynamic Loss Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the fixed Large-Margin Softmax loss with a learned dynamic loss | MLP · MNIST | error rate (%): 1.83 -> 1.69 | Helped |
| replace the fixed Large-Margin Softmax loss with a learned dynamic loss | LeNet · MNIST | error rate (%): 0.88 -> 0.77 | Helped |
| replace the fixed Large-Margin Softmax loss with a learned dynamic loss | ResNet-8, ResNet-20, and ResNet-32 · CIFAR-10 | error rate (%): 11.34 -> 10.82; 8.02 -> 7.63; 7.01 -> 6.95 | Helped |
| replace the fixed Large-Margin Softmax loss with a learned dynamic loss | ResNet-8, ResNet-20, and ResNet-32 · CIFAR-100 | error rate (%): 38.93 -> 38.27; 31.65 -> 30.97; 29.56 -> 29.25 | Helped |
| replace the fixed Large-Margin Softmax loss with a learned dynamic loss | WRN and DenseNet-BC · CIFAR-10 | error rate (%): 3.69 -> 3.42; 3.37 -> 3.08 | Helped |
| replace the fixed Softmax-Margin loss with a learned dynamic loss | LSTM-1, LSTM-2, and Transformer · IWSLT-14 German-English | BLEU: 28.12 -> 29.52; 31.22 -> 31.75; 34.46 -> 34.80 | Helped |
Closest papers by meaning
- L2T-DLN: Learning to Teach with Dynamic Loss Network (NeurIPS 2023 · similarity 0.91)
- Gradient-Based Algorithms for Machine Teaching (CVPR 2021 · similarity 0.88)
- Learning to Teach (ICLR 2018 · similarity 0.88)
- Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · similarity 0.84)
- Can Students Beyond the Teacher? Distilling Knowledge from Teacher’s Bias (AAAI 2025 · similarity 0.84)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.