Research map / Label-efficient and robust learning / Knowledge distillation
Progressive Network Grafting for Few-Shot Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use progressive network grafting instead of KD | VGG16-half · CIFAR10, 5-shot | accuracy: 87.49±1.01 -> 92.88±0.07 | Helped |
| Use progressive network grafting instead of FSKD | VGG16-half · CIFAR10, 1-shot | accuracy: 87.42±0.84 -> 90.74±0.49 | Helped |
| Use progressive network grafting instead of FSKD | VGG16-half · CIFAR100, 5-shot | accuracy: 65.47±0.18 -> 68.16±0.20 | Helped |
| Use progressive network grafting instead of KD | ResNet18 · ILSVRC-2012, 10-shot | Acc@1: 46.40% -> 68.15% | Helped |
| Use progressive network grafting instead of FitNet | ResNet18 · ILSVRC-2012, 10-shot | Acc@5: 73.46% -> 88.83% | Helped |
| Use ResNet18 instead of VGG16-half as the student | Few-shot distillation from VGG16 · CIFAR10, 1-shot | accuracy: 90.74±0.49 -> 73.69±0.91 | Hurt |
| Replace ResNet34 block1 with ResNet18 block1 (ablation) | ResNet34 · ILSVRC-2012 | accuracy: 73.31 -> 72.55 | Hurt |
| Replace ResNet34 block2 with ResNet18 block2 (ablation) | ResNet34 · ILSVRC-2012 | accuracy: 73.31 -> 70.33 | Hurt |
| Replace ResNet34 block3 with ResNet18 block3 (ablation) | ResNet34 · ILSVRC-2012 | accuracy: 73.31 -> 68.11 | Hurt |
| Replace ResNet34 block4 with ResNet18 block4 (ablation) | ResNet34 · ILSVRC-2012 | accuracy: 73.31 -> 68.40 | Hurt |
Closest papers by meaning
- Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks (ICLR 2025 · similarity 0.88)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.88)
- Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process (AAAI 2024 · similarity 0.87)
- All You Need in Knowledge Distillation Is a Tailored Coordinate System (AAAI 2025 · similarity 0.87)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.87)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.