Research map / Label-efficient and robust learning / Knowledge distillation
Collaboration by Competition: Self-coordinated Knowledge Amalgamation for Multi-talent Student Learning
Sihui Luo, Wenwen Pan, Xinchao Wang, Dazhou Wang, Haihong Tang, Mingli Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-modal incorporated amalgamation (MIA) to KD (ablation) | student, depth+edge2d · Edge 2D | rmse: 7.83 -> 6.91 | Improved |
| add adaptive competitive-collaboration training (ACCT) (ablation) | KD + MIA student · Edge 2D | rmse: 6.91 -> 6.40 | Improved |
| add MIA on depth (ablation) | student, depth+edge2d · Depth | rmse: 10.26 -> 10.22 | Improved |
| add ACCT on depth (ablation) | KD + MIA student · Depth | rmse: 10.22 -> 10.21 | Improved |
| add strongly connected auxiliary task keypoint3d (ablation) | student, depth+edge2d · Edge 2D | rmse: 7.83 -> 6.70 | Improved |
| add weakly connected auxiliary task vanishing point (ablation) | student, depth+edge2d · Edge 2D | rmse: 7.83 -> 8.86 | Hurt |
| use SOKA instead of GradNorm supervised multi-task | student, depth+edge2d · Edge 2D | rmse: 7.48 -> 6.40 | Improved |
| use SOKA instead of direct multi-task learning | student, depth+edge2d · Edge 2D | rmse: 8.84 -> 6.40 | Improved |
| distill into compact student instead of teacher PTM | SOKA · Edge 2D | rmse: 6.07 -> 6.40 | Hurt |
| distill into compact student instead of teacher PTM | SOKA · Depth | rmse: 10.22 -> 10.21 | No clear effect |
| start task weights with larger vs smaller learning rate (ablation) | SOKA · depth and edge2d | rmse: 10.30 / 6.37 vs 10.21 / 6.7 | No clear effect |
Closest papers by meaning
- Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation (ICCV 2019 · similarity 0.90)
- Improved Knowledge Distillation via Teacher Assistant (AAAI 2020 · similarity 0.87)
- Deep Mutual Learning (CVPR 2018 · similarity 0.87)
- Amalgamating Multi-Task Models with Heterogeneous Architectures (AAAI 2024 · similarity 0.86)
- Cooperative Knowledge Distillation: A Learner Agnostic Approach (AAAI 2024 · similarity 0.85)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
