Research map / Label-efficient and robust learning / Knowledge distillation
Peer Collaborative Learning for Online Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate peer ensemble and peer mean teachers in PCL | ResNet32 · CIFAR-10 | top-1 error (%): 6.74$\pm$0.15 -> 5.67$\pm$0.12 | Helped |
| Integrate peer ensemble and peer mean teachers in PCL | ResNet110 · CIFAR-100 | top-1 error (%): 23.79$\pm$0.57 -> 20.02$\pm$0.55 | Helped |
| Integrate peer ensemble and peer mean teachers in PCL | ResNet18 · ImageNet ILSVRC 2012 val | top-1 error (%): 30.49$\pm$0.14 -> 29.58$\pm$0.13 | Helped |
| Add peer classification loss (ablation) | Backbone on ResNet110 · CIFAR-100 | top-1 error (%): 23.79$\pm$0.57 -> 23.56$\pm$0.50 | Helped |
| Add ensemble teacher classification loss (ablation) | Backbone + peer classification loss on ResNet110 · CIFAR-100 | top-1 error (%): 23.56$\pm$0.50 -> 23.48$\pm$0.99 | Helped |
| Add peer ensemble distillation loss (ablation) | Backbone + peer and ensemble classification losses on ResNet110 · CIFAR-100 | top-1 error (%): 23.48$\pm$0.99 -> 21.19$\pm$0.62 | Helped |
| Add peer mean teacher distillation loss (ablation) | PCL without peer mean distillation on ResNet110 · CIFAR-100 | top-1 error (%): 21.19$\pm$0.62 -> 20.02$\pm$0.55 | Helped |
| Replace peer mean teacher with mutual distillation (ablation) | PCL on ResNet110 · CIFAR-100 | top-1 error (%): 20.02$\pm$0.55 -> 21.09$\pm$0.18 | Hurt |
| Replace peer mean teacher with traditional weighted mean model (ablation) | PCL on ResNet110 · CIFAR-100 | top-1 error (%): 20.02$\pm$0.55 -> 21.36$\pm$0.73 | Hurt |
| Replace feature-based peer ensemble teacher with logit summation (ablation) | PCL on ResNet110 · CIFAR-100 | top-1 error (%): 20.02$\pm$0.55 -> 20.43$\pm$0.71 | Hurt |
| Remove multiple input augmentation (ablation) | PCL on ResNet110 · CIFAR-100 | top-1 error (%): approximately 0.7% decrease | Hurt |
| Increase the number of branches to four (ablation) | PCL on ResNet110 · CIFAR-100 | top-1 error (%): 19.8% | Helped |
Closest papers by meaning
- Cooperative Knowledge Distillation: A Learner Agnostic Approach (AAAI 2024 · similarity 0.89)
- A Kernel Perspective on Distillation-based Collaborative Learning (NeurIPS 2024 · similarity 0.87)
- How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation (AAAI 2024 · similarity 0.87)
- Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity (NeurIPS 2025 · similarity 0.87)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.87)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.