Research map / Label-efficient and robust learning / Knowledge distillation
Structural Knowledge Distillation for Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace l2 feature distillation loss with SSIM loss | feature KD · RetinaNet R101->R50 | AP: 36.8 -> 40.1 | Helped |
| replace l2 with SSIM loss | feature KD · Faster R-CNN X101->R50 | AP: 37.4 -> 40.9 | Helped |
| replace l2 with l1 loss | feature KD · RetinaNet R50 | AP: 36.8 -> 38.7 | Helped |
| replace l2 with l1 loss | feature KD · Faster R-CNN R50 | AP: 37.4 -> 38.6 | Helped |
| replace prior sampling-based KD with SSIM loss | feature KD · RetinaNet R50 | AP: 38.5 -> 40.1 | Helped |
| replace prior sampling-based KD with SSIM loss (with inheritance) | feature KD · RetinaNet R50 | AP: 40.7 -> 40.7 | No clear effect |
| add structure term to SSIM loss (structure only vs full) (ablation) | l_SSIM · RetinaNet R50 | AP: 39.6 -> 40.1 | Helped |
| add luminance term to contrast+structure SSIM (ablation) | l_SSIM · RetinaNet R50 | AP: 40.0 -> 40.1 | Helped |
| add 1x1 adaptation layer (ablation) | l_SSIM · Cascade RCNN X101->FRCNN R50 | AP: 39.8 -> 40.9 | Helped |
| add 1x1 adaptation layer (ablation) | l_SSIM · RetinaNet R101->R50 | AP: 40.1 -> 40.1 | No clear effect |
| vary SSIM patch size (ablation) | l_SSIM · RetinaNet R101->R50 | AP | No clear effect |
| vary loss weight lambda (ablation) | l_SSIM · RetinaNet R101->R50 | AP: up to +0.5 AP | Mixed |
Closest papers by meaning
- VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.90)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.90)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.89)
- Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation (NeurIPS 2024 · similarity 0.89)
- CrossKD: Cross-Head Knowledge Distillation for Object Detection (CVPR 2024 · similarity 0.88)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.