AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary

Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young Choi

AAAI 2019 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 149 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use BSS-based boundary loss for knowledge distillationStudent classifiers distilled from ResNet26 · CIFAR-10accuracy: 86.66% -> 87.32%Improved
Add BSS distillation to FSPFSP knowledge distillation · CIFAR-10, ResNet8 studentaccuracy: 87.07% -> 87.52%Improved
Add BSS distillation to FSPFSP knowledge distillation · ImageNet 32×32Top5 acc: 57.14% -> 57.27%Improved
Use proposed BSS distillation rather than FITNET+HintonResNet10 student with ResNet42 teacher · TinyImageNetTop1 acc: 53.52% -> 52.99%Hurt
Use BSS samples rather than FGSM samples (ablation)Knowledge distillation · CIFAR-10, ResNet8 studentaccuracy: 87.06% -> 87.32%Improved
Use BSS samples rather than DeepFool samples (ablation)Knowledge distillation · CIFAR-10, ResNet8 studentaccuracy: 86.95% -> 87.32%Improved
Use BSS samples rather than random noise (ablation)Knowledge distillation · CIFAR-10, ResNet8 studentaccuracy: 86.73% -> 87.32%Improved
Choose base samples with the proposed selection rule rather than using all samples (ablation)BSS distillation · CIFAR-10, ResNet8 studentaccuracy: 87.18% -> 87.32%Improved
Choose base samples with the proposed selection rule rather than randomly (ablation)BSS distillation · CIFAR-10, ResNet8 studentaccuracy: 87.10% -> 87.32%Improved
Sample target classes according to teacher class probabilities rather than randomly (ablation)BSS distillation · CIFAR-10, ResNet8 studentaccuracy: 87.08% -> 87.32%Improved
Use BSS distillation rather than classification-only trainingResNet8 student · CIFAR-10accuracy: 86.02% -> 87.32%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy