Research map / Label-efficient and robust learning / Knowledge distillation
Knowledge Distillation via Route Constrained Optimization
Xiao Jin, Baoyun Peng, Yichao Wu, Yu Liu, Jiaheng Liu, Ding Liang, Junjie Yan, Xiaolin Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add route-constrained distillation (RCO) instead of KD | MobileNetV2 · CIFAR-100 | top-1 accuracy: 68.71 -> 70.85 | Improved |
| add RCO instead of KD | MobileNetV2 · ImageNet val | top-1 accuracy: 66.75 -> 68.21 | Improved |
| add RCO instead of FitNet hint learning | MobileNetV2 · MegaFace 1M distractors | top-1 accuracy: 81.02 -> 84.3 | Improved |
| use one-stage EEI with M_gap 10 instead of KD (ablation) | MobileNetV2 T=4 w=0.35 · CIFAR-100 | top-1 accuracy: 60.79 -> 65.01 | Improved |
| use RCO M_gap 10 instead of DML (ablation) | MobileNetV2 T=4 w=0.35 · CIFAR-100 | top-1 accuracy: 61.13 -> 65.01 | Improved |
| use M_gap 10 instead of per-iteration mimicking (ablation) | RCO one-stage · CIFAR-100 | top-1 accuracy: 61.63 -> 65.01 | Improved |
| use M_gap 20 instead of 10 (ablation) | RCO one-stage · CIFAR-100 | top-1 accuracy: 65.01 -> 63.88 | Hurt |
| use greedy search anchors instead of EEI-4 (ablation) | RCO multi-stage · CIFAR-100 | top-1 accuracy: 65.27 -> 65.41 | Improved |
| use EEI-2 instead of one-stage EEI (ablation) | RCO · CIFAR-100 | top-1 accuracy: 65.01 -> 61.43 | Hurt |
| use more anchor points: EEI-3 vs EEI-2 (ablation) | RCO multi-stage · CIFAR-100 | top-1 accuracy: 61.43 -> 63.34 | Improved |
| use RCO on smaller students | MobileNetV2 variants · CIFAR-100 | top-1 accuracy | Improved |
| add RCO instead of KD for noise robustness | MobileNetV2 · CIFAR-100 noisy test | loss gap | Improved |
Closest papers by meaning
- Teacher Guided Neural Architecture Search for Face Recognition (AAAI 2021 · similarity 0.89)
- SDPGO: Efficient Self-Distillation Training Meets Proximal Gradient Optimization (NeurIPS 2025 · similarity 0.89)
- PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning (ICML 2026 · similarity 0.87)
- Adaptive Dual Guidance Knowledge Distillation (AAAI 2025 · similarity 0.87)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.87)
- Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
