Research map / Label-efficient and robust learning / Knowledge distillation
Robust Knowledge Transfer via Hybrid Forward on the Teacher-Student Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add hybrid forward transfer from ResNet-50 with dynamic balancing | MobileNet-v1 · ImageNet | Top-1: 68.87 -> 71.39 | Helped |
| Add hybrid forward transfer from ResNet-50 with dynamic balancing | MobileNet-v1 · ImageNet | Top-5: 88.76 -> 90.47 | Helped |
| Add hybrid forward transfer from a GTA5 segmentation teacher | CityScapes segmentation student · CityScapes val | mIoU: 59.7 -> 60.7 | Helped |
| Add hybrid forward transfer from a CityScapes detection teacher | CityScapes segmentation student · CityScapes val | mIoU: 59.7 -> 60.2 | Helped |
| Use the GTA5 segmentation teacher instead of the CityScapes detection teacher (ablation) | CityScapes segmentation student · CityScapes val | mIoU: 60.2 -> 60.7 | Helped |
| Add hybrid forward transfer from the GTA5 segmentation teacher | CityScapes segmentation student · GTA5 val | mIoU: 33.5 -> 34.2 | Helped |
| Add hybrid forward transfer from the CityScapes detection teacher | CityScapes segmentation student · GTA5 val | mIoU: 33.5 -> 33.6 | No clear effect |
| Add hybrid forward transfer from a human pose estimation teacher | Faster RCNN with MobileNet v1 on CityPersons · CityPersons val, R | miss-rate: 17.20 -> 16.19 | Helped |
| Add hybrid forward transfer from a human pose estimation teacher | Faster RCNN with MobileNet v1 on CityPersons · CityPersons val, R small | miss-rate: 21.87 -> 21.12 | Helped |
| Add hybrid forward transfer from a human pose estimation teacher | Faster RCNN with MobileNet v1 on CityPersons · CityPersons val, R occ heavy | miss-rate: 61.76 -> 64.11 | Hurt |
| Add hybrid forward transfer from a human pose estimation teacher | Faster RCNN with MobileNet v1 on CityPersons · CityPersons val, All | miss-rate: 47.09 -> 46.85 | Helped |
| Replace dynamic balancing with a manually fixed balancing parameter (ablation) | Hybrid forward on CityPersons pedestrian detection · CityPersons val | miss-rate | Hurt |
Closest papers by meaning
- A Gift From Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning (CVPR 2017 · similarity 0.89)
- Improved Knowledge Distillation via Teacher Assistant (AAAI 2020 · similarity 0.89)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.88)
- Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model (ICLR 2024 · similarity 0.88)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.87)
- Masking Teacher and Reinforcing Student for Distilling Vision-Language Models (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.