Research map / Label-efficient and robust learning / Knowledge distillation
Dynamic Self-Supervised Teacher-Student Network Learning
Fei Ye, Adrian G. Borş
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dynamic expandable Teacher with KDS instead of a single GAN Teacher (LTS) | Student · CCCSSM lifelong learning | MSE: 161.49 -> 115.35 | Improved |
| use dynamic Teacher (D-TS-KFD) instead of LTS | Student · MSFIR lifelong learning | MSE: 44.83 -> 31.17 | Improved |
| use NLL expert-selection criterion instead of KFD (ablation) | D-TS · CCCSSM lifelong learning | MSE: 115.35 -> 105.43 | Improved |
| use KFD criterion instead of NLL (ablation) | D-TS · MSFIR lifelong learning | MSE: 34.07 -> 31.17 | Improved |
| add conditional prior regularized by variable u in KL term (ablation) | D-TS-KFD · Inter-domain interpolation | MSE: 224.28 -> 210.84 | Improved |
| add dynamic Teacher expansion (D-TS-KFD) instead of LTS | Student · Missing training data CCCSSM | MSE: 241.23 -> 118.83 | Improved |
| update existing expert for 5 epochs instead of 20 (ablation) | D-TS-KFD · CCCSSM lifelong learning | MSE: 115.35 -> 116.6 | No clear effect |
| lower expansion threshold hold (ablation) | D-TS-KFD · CCCSSM lifelong learning | MSE | Improved |
| use D-TS-KFD instead of BatchEnsemble Student BE-Stu | Student · Supervised LLL average | accuracy (%): 76.86 -> 82.09 | Improved |
| use D-TS-KFD instead of EWC | Student · Supervised LLL average | accuracy (%): 75.35 -> 82.09 | Improved |
| use NLL instead of KFD criterion (ablation) | D-TS · Supervised LLL average | accuracy (%): 82.09 -> 79.88 | Hurt |
| use D-TS-KFD instead of LGM | Student · Six-task MFSICO | MSE: 243.66 -> 87.42 | Improved |
Closest papers by meaning
- Lifelong Teacher-Student Network Learning (TPAMI 2021 · similarity 0.90)
- How a Student becomes a Teacher: learning and forgetting through Spectral methods (NeurIPS 2023 · similarity 0.87)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.86)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.85)
- Multi-Teacher Knowledge Distillation with Reinforcement Learning for Visual Recognition (AAAI 2025 · similarity 0.85)
- Scaling Multi-Teacher Distillation for Digital Pathology (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
