Research map / Label-efficient and robust learning / Knowledge distillation
Learning Efficient Object Detection Models with Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add knowledge distillation and hint learning from a VGG16 teacher to Tucker student | Tucker student (Faster-RCNN) · PASCAL VOC 2007 | mAP@0.5: +4.7 | Helped |
| add distillation and hint learning from VGG16 teacher | Tucker student · KITTI | mAP@0.5: +4.4 | Helped |
| add distillation with deeper teacher | student models across datasets · PASCAL, KITTI, ILSVRC, COCO | mAP@0.5: +1.2 to +4.7 across pairs | Helped |
| halve input resolution for student while distilling from high-res teacher | AlexNet student · PASCAL VOC 2007 | mAP@0.5: +3.5 over low-res baseline (56.7 vs 53.2), near high-res teacher 57.2 at ~2x speed | Helped |
| apply distillation at 20% FLOPS budget of Tucker AlexNet | Tucker AlexNet compressed student · PASCAL VOC 2007 | mAP@0.5: +5.2 | Helped |
| add weighted cross-entropy distillation loss (CLS-W) instead of standard CE (ablation) | Tucker student classification · PASCAL VOC 2007 | mAP@0.5: +3.0 (57.7 vs 54.7 baseline) | Helped |
| teacher bounded regression loss (L2-B) instead of plain L2 regression (ablation) | Tucker student regression · PASCAL VOC 2007 | mAP@0.5: +1.3 over L2 (55.9 vs 54.6) | Helped |
| teacher bounded regression loss (L2-B) instead of plain L2 regression (ablation) | Tucker student regression · KITTI | mAP@0.5: +1.6 over L2 (50.1 vs 48.5) | Helped |
| add adaptation layer before hint loss (ablation) | hint learning in Tucker student · PASCAL VOC 2007 | mAP@0.5: +1.1 (58.0 vs 56.9) | Helped |
| add adaptation layer before hint loss (ablation) | hint learning in Tucker student · KITTI | mAP@0.5: +1.8 (52.1 vs 50.3) | Helped |
| add hint learning without adaptation layer (ablation) | Tucker student · PASCAL VOC 2007 | mAP@0.5: +2.2 (56.9 vs 54.7) | Helped |
| add hint learning (with adaptation) on top of bounded regression and weighted CE (ablation) | Tucker student · PASCAL VOC 2007 | mAP@0.5: +1.0 (59.4 vs 58.4) | Helped |
Closest papers by meaning
- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors (ICLR 2021 · similarity 0.94)
- Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation (ICML 2023 · similarity 0.92)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.90)
- CrossKD: Cross-Head Knowledge Distillation for Object Detection (CVPR 2024 · similarity 0.90)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.90)
- VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.