Research map / Label-efficient and robust learning / Knowledge distillation
Adaptive Hierarchy-Branch Fusion for Online Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hierarchical branches and adaptive hierarchy-branch fusion instead of the strongest reported baseline | ResNet32 · CIFAR-10 | error (%): 5.49 ± 0.10 -> 5.32 ± 0.11 | Helped |
| use hierarchical branches and adaptive hierarchy-branch fusion instead of the strongest reported baseline | ResNet110 · CIFAR-10 | error (%): 4.48 ± 0.10 -> 4.33 ± 0.08 | Helped |
| use hierarchical branches and adaptive hierarchy-branch fusion instead of the strongest reported baseline | ResNet32 · CIFAR-100 | error (%): 25.50 ± 0.10 -> 25.19 ± 0.16 | Helped |
| use hierarchical branches and adaptive hierarchy-branch fusion instead of the strongest reported baseline | ResNet110 · CIFAR-100 | error (%): 21.14 ± 0.14 -> 20.96 ± 0.14 | Helped |
| use AHBF-OKD instead of OKDDip | ResNet18 · ImageNet 2012 | Top-1 error (%): 29.37* -> 29.28 | Helped |
| use AHBF-OKD instead of OKDDip | ResNet34 · ImageNet 2012 | Top-1 error (%): 25.60 -> 25.47 | Helped |
| add hierarchical teacher-assistant distillation losses (LTAKL and LBTKL) (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%): without hierarchical teacher-assistant distillation -> 25.19%; LFKL surpasses only LTAKL by 1.13%; only LBTKL by 1.40% | Helped |
| distill between adjacent teacher assistants using LTAKL (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%): 27.04% (λ1=1, λ2=0) vs 27.19% (λ1=0, λ2=1) | Helped |
| set both hierarchical distillation loss weights to zero (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%): 28.88 | Hurt |
| increase the ramp-up parameter E from 80 to 300 (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%): 300 surpasses 80 by 1.02%; lowest error 25.19% | Helped |
| remove auxiliary layers so all branches have the same structure (a=0) (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%): 26.47 -> 25.19 | Hurt |
| increase the number of hierarchical branches beyond four (ablation) | AHBF-OKD ResNet32 · CIFAR-100 | error (%) | Hurt |
Closest papers by meaning
- Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity (NeurIPS 2025 · similarity 0.89)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.88)
- CIFD: Controlled Information Flow to Enhance Knowledge Distillation (NeurIPS 2024 · similarity 0.87)
- How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation (AAAI 2024 · similarity 0.87)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.87)
- Adaptive Dual Guidance Knowledge Distillation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.