Research map / Label-efficient and robust learning / Knowledge distillation
Learning to Reweight with Deep Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use student internal features and label embeddings to set training weights | Learning to Reweight with Deep Interactions · CIFAR-10 and CIFAR-100 clean | test error rate: 7.22 -> 6.20; 29.57 -> 28.65 (ResNet-32) | Helped |
| Use student internal features and label embeddings to set training weights | Learning to Reweight with Deep Interactions · IWSLT'14 De→En noisy | BLEU: 33.68 -> 35.56 | Helped |
| Add internal features to surface-feature-based reweighting (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.54 -> 6.20 | Helped |
| Add label embeddings to internal-feature-based reweighting (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.34 -> 6.20 | Helped |
| Add surface features M₁ to I₀ + M₀ (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.30 | Hurt |
| Replace second-to-last-layer features with last convolutional layer features (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.22; 6.20 -> 6.31; 6.20 -> 6.21 | No clear effect |
| Add one hidden layer to the simplest teacher network (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.48 | Hurt |
| Add two hidden layers to the simplest teacher network (ablation) | Teacher model on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.59 | Hurt |
| Allow gradients to flow through the sample weights into the student (ablation) | Teacher and student on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.08 | Helped |
| Use K=1, B=1 instead of default K=20, B=2 (ablation) | Teacher optimization on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.56 | Hurt |
| Use K=20, B=5 instead of default K=20, B=2 (ablation) | Teacher optimization on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.41 | Hurt |
| Use K=100, B=2 instead of default K=20, B=2 (ablation) | Teacher optimization on CIFAR-10 clean · CIFAR-10 clean | test error rate: 6.20 -> 6.30 | Hurt |
Closest papers by meaning
- Learning to Reweight Examples for Robust Deep Learning (ICML 2018 · similarity 0.89)
- Improving Generalization via Meta-Learning on Hard Samples (CVPR 2024 · similarity 0.85)
- Thumb on the Scale: Optimal Loss Weighting in Last Layer Retraining (NeurIPS 2025 · similarity 0.85)
- Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling (AAAI 2025 · similarity 0.85)
- Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining (ICLR 2025 · similarity 0.84)
- Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.