AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Knowledge Distillation via Instance Relationship Graph

Yufan Liu, Jiajiong Cao, Bing Li, Chunfeng Yuan, Weiming Hu, Yangxi Li, Yunqiang Duan

CVPR 2019 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 323 citations (OpenAlex)

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Distill instance features and instance relationships using IRG (ablation)ResNet20-x0.5 · CIFAR10accuracy: 88.36 -> 90.28Improved
Add IRG transformation distillation to LIRG (ablation)ResNet20-x0.5 · CIFAR10accuracy: 90.28 -> 90.69Improved
Use one-to-many instead of one-to-one layer supervision (ablation)LIRG with ResNet20-x0.5 student and ResNet20 teacher · CIFAR10accuracy: 90.03 -> 90.28Improved
Use one-to-many instead of one-to-one layer supervision (ablation)LIRG with ResNet20-x0.5 student and ShuffleNet-x0.5 teacher · CIFAR10accuracy: 89.89 -> 90.21Improved
Use one-to-many supervision with three student layers rather than one (ablation)LIRG with ResNet20-x0.5 student and ResNet20 teacher · CIFAR10accuracy: 89.87 -> 90.28Improved
Use one-to-many supervision with five student layers rather than three (ablation)LIRG with ResNet20-x0.5 student and ResNet20 teacher · CIFAR10accuracy: 90.28 -> 90.02Hurt
Add IRG distillation rather than training only with ground-truth loss (ablation)ResNet20-x0.5 · CIFAR100-coarseaccuracy: 72.51 -> 74.32Improved
Add IRG transformation distillation to LIRG (ablation)ResNet20-x0.5 · CIFAR100-coarseaccuracy: 74.32 -> 74.64Improved
Add IRG distillation rather than training only with ground-truth loss (ablation)ResNet20-x0.5 · CIFAR100-fineaccuracy: 59.88 -> 61.93Improved
Add IRG transformation distillation to LIRG (ablation)ResNet20-x0.5 · CIFAR100-fineaccuracy: 61.93 -> 62.25Improved
Use LMTK rather than Rocket distillationResNet14-x0.5 · CIFAR10-smallaccuracy: 62.23 -> 66.04Improved
Use LMTK rather than Rocket distillationResNet20-x0.375 · ImageNetaccuracy: 76.16 -> 77.18Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy