Research map / Label-efficient and robust learning / Knowledge distillation
Pixel Distillation: Cost-Flexible Distillation Across Image Sizes and Heterogeneous Networks
Guangyu Guo, Dingwen Zhang, Longfei Han, Nian Liu, Ming‐Ming Cheng, Junwei Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KD to baseline student (ablation) | Baseline-FT · CUB (setting b, K=4) | top-1 accuracy: 41.56 -> 49.38 | Improved |
| add ISRD to KD one-stage baseline (ablation) | vanilla PD · CUB (setting b, K=4) | top-1 accuracy: 49.38 -> 50.34 | Improved |
| add TAS assistant stage (ablation) | vanilla PD · CUB (setting b, K=4) | top-1 accuracy: 50.34 -> 51.40 | Improved |
| add ICF feature distillation in input compression stage (ablation) | TAS · CUB (setting b, K=4) | top-1 accuracy: 51.40 -> 52.44 | Improved |
| add ISRD to TAS-AFP (CrossKD) detector (ablation) | TAS-AFP (CrossKD) · PASCAL VOC (setting h, K=4) | mAP: 61.3 -> 62.9 | Improved |
| add Cao et al. FPN feature distillation (ablation) | TAS-AFP (CrossKD) · COCO (setting i, K=4) | mAP: 25.1 -> 24.6 | Hurt |
| add Cao et al. FPN feature distillation | TAS-AFP (CrossKD) · COCO (setting i, K=2) | mAP: 33.7 -> 35.0 | Improved |
| add AFP and TAS to student detector | RetinaNet student · PASCAL VOC (setting h, K=4) | mAP: 56.9 -> 61.3 | Improved |
| use two-stage TAS instead of vanilla PD with DKD | vanilla PD (DKD) · ImageNet (setting b, K=4) | top-1 accuracy: 52.84 -> 53.83 | Improved |
| use feature distillation AT at K=4 | KD student · CUB (setting b, K=4) | top-1 accuracy: 49.38 -> 4.55 | Hurt |
| do model compression before input compression in TAS (ablation) | TAS · CUB (settings b and f) | top-1 accuracy | Improved |
| increase ISRD decoder kernel size from 1x1 to 7x7 (ablation) | ISRD · CUB (setting b) | top-1 accuracy | Hurt |
Closest papers by meaning
- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors (ICLR 2021 · similarity 0.91)
- Structured Knowledge Distillation for Accurate and Efficient Object Detection (TPAMI 2023 · similarity 0.91)
- Distilling Image Classifiers in Object Detectors (NeurIPS 2021 · similarity 0.89)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.89)
- Fuse Before Transfer: Knowledge Fusion for Heterogeneous Distillation (ICCV 2025 · similarity 0.89)
- VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
