AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Differentiable Feature Aggregation Search for Knowledge Distillation

Yushuo Guan, Pengyu Zhao, Bingxuan Wang, Yuanxing Zhang, Cong Yao, Kaigui Bian, Jian Tang

ECCV 2020 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 45 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use searched feature aggregation instead of KD output distillationWRN_16_4 student · CIFAR-100accuracy (%): 78.4 -> 79.74Improved
use searched aggregation instead of last-layer-only features (Last) (ablation)WRN_16_4 student · CIFAR-100accuracy (%): 79.11 -> 79.74Improved
use searched aggregation instead of last-layer-only features (Last) (ablation)WRN_28_2 student · CIFAR-100accuracy (%): 77.86 -> 78.18Improved
use searched aggregation instead of last-layer-only features (Last) (ablation)WRN_16_2 student · CIFAR-100accuracy (%): 75.51 -> 75.85Improved
use searched aggregation instead of random weights (ablation)WRN_16_4 student · CIFAR-100accuracy (%): 78.92 -> 79.74Improved
use searched aggregation instead of random weights (ablation)WRN_16_2 student · CIFAR-100accuracy (%): 74.25 -> 75.85Improved
use searched aggregation instead of average weights (ablation)WRN_16_4 student · CIFAR-100accuracy (%): 78.81 -> 79.74Improved
use searched aggregation instead of average weights (ablation)WRN_28_2 student · CIFAR-100accuracy (%): 76.99 -> 78.18Improved
add DFA feature distillation over SPWRN_28_2 student · CIFAR-100accuracy (%): 77.41 -> 78.18Improved
search aggregation weights on CIFAR-100 instead of CINIC-10 (DFA-T) (ablation)DFA, ShuffleNetV2 x=1.0 -> x=0.5 · CINIC-10accuracy (%): 79.97 -> 79.38Hurt
search aggregation weights on CIFAR-100 instead of CINIC-10 (DFA-T) (ablation)DFA, ShuffleNetV2 x=2.0 -> x=1.0 · CINIC-10accuracy (%): 85.38 -> 85.41No clear effect
search aggregation weights on CIFAR-100 instead of CINIC-10 (DFA-T) (ablation)DFA, ShuffleNetV2 x=2.0 -> x=0.5 · CINIC-10accuracy (%): 79.51 -> 79.45No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy