AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Dynamic Self-Supervised Teacher-Student Network Learning

Fei Ye, Adrian G. Borş

TPAMI 2022 · Label-efficient and robust learning › Knowledge distillation · approach: Teacher properties and robustness · 22 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use dynamic expandable Teacher with KDS instead of a single GAN Teacher (LTS)Student · CCCSSM lifelong learningMSE: 161.49 -> 115.35Improved
use dynamic Teacher (D-TS-KFD) instead of LTSStudent · MSFIR lifelong learningMSE: 44.83 -> 31.17Improved
use NLL expert-selection criterion instead of KFD (ablation)D-TS · CCCSSM lifelong learningMSE: 115.35 -> 105.43Improved
use KFD criterion instead of NLL (ablation)D-TS · MSFIR lifelong learningMSE: 34.07 -> 31.17Improved
add conditional prior regularized by variable u in KL term (ablation)D-TS-KFD · Inter-domain interpolationMSE: 224.28 -> 210.84Improved
add dynamic Teacher expansion (D-TS-KFD) instead of LTSStudent · Missing training data CCCSSMMSE: 241.23 -> 118.83Improved
update existing expert for 5 epochs instead of 20 (ablation)D-TS-KFD · CCCSSM lifelong learningMSE: 115.35 -> 116.6No clear effect
lower expansion threshold hold (ablation)D-TS-KFD · CCCSSM lifelong learningMSEImproved
use D-TS-KFD instead of BatchEnsemble Student BE-StuStudent · Supervised LLL averageaccuracy (%): 76.86 -> 82.09Improved
use D-TS-KFD instead of EWCStudent · Supervised LLL averageaccuracy (%): 75.35 -> 82.09Improved
use NLL instead of KFD criterion (ablation)D-TS · Supervised LLL averageaccuracy (%): 82.09 -> 79.88Hurt
use D-TS-KFD instead of LGMStudent · Six-task MFSICOMSE: 243.66 -> 87.42Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy