AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

A Good Teacher Adapts Their Knowledge for Distillation

Chengyao Qian, Trung Le, Mehrtash Harandi

ICCV 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Distillation with causal and adaptive guidance · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add teacher fine-tuning (AID) before KDvanilla KD, ResNet110x4 teacher · CIFAR-100, ResNet8 studenttop-1 accuracy: 61.47 -> 63.44Improved
add teacher fine-tuning (AID) before KDvanilla KD, ResNet110x4 teacher · CIFAR-100, ResNet14 studenttop-1 accuracy: 66.96 -> 69.73Improved
add teacher fine-tuning (AID) before KDvanilla KD, ResNet110x4 teacher · CIFAR-100, WRN-16-2 studenttop-1 accuracy: 75.57 -> 76.68Improved
use larger teacher (ResNet32x4 instead of ResNet56) with vanilla KDKD, ResNet20 student · CIFAR-100top-1 accuracy: 70.66 -> 69.83Hurt
use larger teacher (ResNet32x4 instead of ResNet56) with AIDAID, ResNet20 student · CIFAR-100top-1 accuracy: 72.50 -> 72.63Improved
distill only class-wise similarity term (ablation)KD, ResNet20 student · CIFAR-100 (Figure 2)top-1 accuracyNo clear effect
distill only intra-class distribution term (ablation)KD, ResNet20 student · CIFAR-100 (Figure 2)top-1 accuracyHurt
fine-tune teacher (ResNet32x4) with a different smaller student, ResNet20 student (ablation)AID · CIFAR-100top-1 accuracy: 72.63 -> 72.43 across MFT choicesImproved
fine-tune teacher with larger model WRN-40-2 (ResNet20 student, ResNet32x4 teacher) (ablation)AID · CIFAR-100top-1 accuracy: 70.04No clear effect
fine-tune ResNet32x4 teacher with ResNet20 instead of the target student for WRN-40-1 (ablation)AID · CIFAR-100top-1 accuracy: 75.77 -> 75.08Hurt
use AID instead of retraining teacher/assistants (run-time)TAKD · CIFAR-100 run-timeminutes: 699.8 -> 179.4Improved
add teacher fine-tuning (AID)STD, ResNet110x4 teacher, ResNet8 · CIFAR-100top-1 accuracy: 59.43 -> 63.44Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy