AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Distilling Global and Local Logits With Densely Connected Relations

Youmin Kim, Jinbae Park, YounHo Jang, Muhammad Ali, Tae-Hyun Oh, Sung-Ho Bae

ICCV 2021 · Label-efficient and robust learning › Knowledge distillation · approach: Knowledge distillation for detectors · 42 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace KD with GLD (global + local logits + relations)KD student WRN16-2 · CIFAR-100 val (WRN22-4 -> WRN16-2)top-1 accuracy: 74.75 -> 76.28Improved
replace KD with GLDResNet18 student · ImageNet valtop-1 accuracy: 70.66 -> 71.63Improved
distill with GLD instead of no distillation, then transfer to CINIC-10WRN16-1 student · CIFAR-100 to CINIC-10 transfertop-1 accuracy: 58.36 -> 61.09Improved
distill with GLD for SSD detector backboneSSD-ResNet18 · COCO2017 val detectionmAP: 19.41 -> 19.83Improved
distill with GLD for DeepLabV3+ backboneDLV3+-ResNet18 · COCO2017 val segmentationmIoU: 51.22 -> 53.75Improved
use standard deviation instead of temperature 4 as softening factor, global logit only (ablation)KD (global logit only) · CIFAR-100 val (case c)top-1 accuracy: 74.75 -> 74.30Hurt
use standard deviation instead of temperature 4 as softening factor, global+local logits (ablation)GLD without relation loss · CIFAR-100 val (cases a,b,c,d)top-1 accuracy: 69.77 -> 71.02 (a), 67.44 -> 68.85 (b), 75.06 -> 74.89 (c), 72.50 -> 74.25 (d)Mixed
increase local logits from 2x2 to 7x7 on fine-grained CUB-200-2011 (ablation)GLD · CUB-200-2011top-1 accuracy: 68.89 -> 70.81Improved
increase local logits from 2x2 to 7x7 on ImageNet (ablation)GLD · ImageNet valtop-1 accuracy: 71.63 -> 69.95Hurt
increase local logits from 2x2 to 8x8 on CIFAR-100 (case b) (ablation)GLD · CIFAR-100 val (WRN16-2 -> WRN16-1)top-1 accuracy: 68.95 -> 65.66Hurt
vary alpha in 0.1-0.7 and beta in 100-1000 (ablation)GLD · CIFAR-100 val (case c)top-1 accuracy: 76.00 to 76.28No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy