Research map / Label-efficient and robust learning / Knowledge distillation
AMLN: Adversarial-based Mutual Learning Network for Online Knowledge Distillation
Xiaobing Zhang, Shijian Lu, Haigang Gong, Zhipeng Luo, Ming Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add block-wise intermediate-to-final feature alignment loss L_F on top of CE+KL (ablation) | AMLN ResNet32 peers · CIFAR-100 | top-1 accuracy: 70.98 -> 73.24 | Improved |
| add block-wise intermediate-to-final feature alignment loss L_F on top of CE+KL (ablation) | AMLN ResNet32 peers · CIFAR-10 | top-1 accuracy: 93.15 -> 93.89 | Improved |
| add adversarial block-wise discriminator loss L_D on top of CE+KL (ablation) | AMLN ResNet32 peers · CIFAR-100 | top-1 accuracy: 70.98 -> 74.16 | Improved |
| add adversarial block-wise discriminator loss L_D on top of CE+KL (ablation) | AMLN ResNet32 peers · CIFAR-10 | top-1 accuracy: 93.15 -> 94.01 | Improved |
| add L_D and L_F together to outcome-driven losses (ablation) | AMLN ResNet32 peers · CIFAR-100 | top-1 accuracy: 70.98 -> 74.69 | Improved |
| add L_D and L_F together to outcome-driven losses (ablation) | AMLN ResNet32 peers · CIFAR-10 | top-1 accuracy: 93.15 -> 94.25 | Improved |
| add L_F on top of L_D (full vs D only) (ablation) | AMLN ResNet32 peers · CIFAR-100 | top-1 accuracy: 74.16 -> 74.69 | Improved |
| add KL mutual distillation to CE (DML-like) (ablation) | ResNet32 peers · CIFAR-10 | top-1 accuracy: 93.10 -> 93.15 | No clear effect |
| add KL mutual distillation to CE (DML-like) (ablation) | ResNet32 peers · CIFAR-100 | top-1 accuracy: 69.71 -> 70.98 | Improved |
| replace FFL with AMLN online distillation | ResNet34 peers · ImageNet | top-1 accuracy: 74.42 -> 75.48 | Improved |
| replace FFL with AMLN online distillation | ResNet32 peers · CIFAR-100 | top-1 accuracy: 72.97 -> 74.69 | Improved |
| replace FFL with AMLN online distillation | WRN-40-2 peers · CIFAR-10 | top-1 accuracy: 95.17 -> 95.21 | Improved |
Closest papers by meaning
- Deep Mutual Learning (CVPR 2018 · similarity 0.88)
- Peer Collaborative Learning for Online Knowledge Distillation (AAAI 2021 · similarity 0.87)
- Weighted Mutual Learning with Diversity-Driven Model Compression (NeurIPS 2022 · similarity 0.87)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.87)
- A Kernel Perspective on Distillation-based Collaborative Learning (NeurIPS 2024 · similarity 0.86)
- Adversarial Moment-Matching Distillation of Large Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
