Research map / Label-efficient and robust learning / Knowledge distillation
PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MSE loss with PCC (normalized features) (ablation) | FPN feature distillation · FCOS-ResX101 -> Retina-Res50 2x | mAP: 36.3 -> 40.8 | Helped |
| replace MSE with PCC (ablation) | FPN feature distillation · GFL-Res101 -> FCOS-Res50 2x | mAP: 43.0 -> 43.5 | Helped |
| add PKD normalization to FitNet (ablation) | FitNet · GFL101 -> GFL50 | mAP: 41.3 -> 43.3 | Helped |
| add PKD to FGD global (ablation) | FGD Global · GFL101 -> GFL50 | mAP: 40.2 -> 43.3 | Helped |
| add PKD to GT Mask imitation (ablation) | GT Mask · GFL101 -> GFL50 | mAP: 41.4 -> 43.0 | Helped |
| add PKD to FRS feature imitation (ablation) | FRS · GFL101 -> GFL50 | mAP: 43.2 -> 43.4 | Helped |
| replace Pearson-normalized MSE with softmax KL on normalized features (ablation) | PKD · Retina-ResX101 -> Retina-Res50 | mAP: 40.8 -> 40.9 | No clear effect |
| vary loss weight alpha from 3 to 13 (ablation) | PKD · FCOS-ResX101 -> Retina-Res50 | mAP: 41.0 to 41.3 | No clear effect |
| use stronger heterogeneous teacher (Mask RCNN-Swin vs Retina-Res101) (ablation) | PKD · Retina-Res50 | mAP: 39.6 -> 41.5 | Helped |
| replace backbone and neck of FCOS with GFL-trained ones (ablation) | FCOS · COCO | mAP: 36.5 -> 37.6 | Helped |
| replace backbone and neck of RetinaNet with FCOS-trained ones (ablation) | RetinaNet · COCO | mAP: 36.3 -> 35.2 | Hurt |
| use PKD instead of FGD | FCOS-Res50 student · COCO | mAP: 42.3 -> 42.8 | Helped |
Closest papers by meaning
- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors (ICLR 2021 · similarity 0.92)
- Focal and Global Knowledge Distillation for Detectors (CVPR 2022 · similarity 0.92)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.92)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.92)
- CrossKD: Cross-Head Knowledge Distillation for Object Detection (CVPR 2024 · similarity 0.90)
- DetKDS: Knowledge Distillation Search for Object Detectors (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.