AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition

Chuanguang Yang, Zhulin An, Helong Zhou, Fuzhen Zhuang, Yongjun Xu, Qian Zhang

TPAMI 2023 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 119 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add instance-level contrastive loss ICL (ablation)L-MCL · CIFAR-100 ResNet-110top-1 accuracy (%): 75.29 -> 78.34Improved
add instance-level contrastive loss ICL (ablation)L-MCL · CIFAR-100 WRN-40-2top-1 accuracy (%): 76.89 -> 79.29Improved
add conventional contrastive loss VCL (ablation)L-MCL · CIFAR-100 ResNet-110top-1 accuracy (%): 75.29 -> 77.35Improved
add conventional contrastive loss VCL (ablation)L-MCL · CIFAR-100 WRN-40-2top-1 accuracy (%): 76.89 -> 77.66Improved
add soft contrastive loss terms to ICL (ablation)L-MCL · CIFAR-100 ResNet-110top-1 accuracy (%): 78.34 -> 78.83Improved
add all four contrastive terms (ablation)L-MCL · CIFAR-100 ResNet-110top-1 accuracy (%): 78.83 -> 79.05Improved
add L-MCL to logit-level online KD (ablation)logit OKD · CIFAR-100 ResNet-110top-1 accuracy (%): 77.13 -> 79.48Improved
add L-MCL to logit-level online KD (ablation)logit OKD · CIFAR-100 WRN-40-2top-1 accuracy (%): 78.64 -> 80.96Improved
use weighted all-to-all layer matching instead of one-to-one (ablation)L-MCL · CIFAR-100 ShuffleV2-ResNet-110 (ResNet-110)top-1 accuracy (%): 77.12 -> 78.85Improved
use weighted all-to-all instead of all-to-all layer matching (ablation)L-MCL · CIFAR-100 ShuffleV2-ResNet-110 (ShuffleV2)top-1 accuracy (%): 75.29 -> 76.46Improved
replace logit-based online KD with L-MCLonline KD · STL-10 transfer (WRN-40-2)top-1 accuracy (%): 62.11 -> 67.86Improved
apply logit-based online KD methods (DML)DML · STL-10 transfer (WRN-40-2)top-1 accuracy (%): 62.11 -> 61.02Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy