Research map / Label-efficient and robust learning / Knowledge distillation
MEAL: Multi-Model Ensemble via Adversarial Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the student with MEAL rather than the single-model baseline | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.34 -> 5.55 | Helped |
| Train the student with MEAL rather than the single-model baseline | Shake-Shake-26 2x96d · CIFAR-10 test | error rate (%): 2.86 -> 2.54 | Helped |
| Use MEAL instead of Dropout training | CIFAR-10 models · CIFAR-10 test | error rate (%): 6.89 -> 5.55; 5.37 -> 4.83; 4.69 -> 4.35; 3.75 -> 3.54 | Helped |
| Distill stronger teacher knowledge into ResNet-50 with MEAL Plus | ResNet-50 · ImageNet val | Top-1 error (%): 23.85 -> 21.79 | Helped |
| Distill stronger teacher knowledge into ResNet-50 with MEAL Plus | ResNet-50 · ImageNet val | Top-5 error (%): 7.13 -> 5.99 | Helped |
| Use L1 similarity loss rather than the baseline training (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.34 -> 6.97 | Hurt |
| Use L2 similarity loss only rather than the baseline training (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.34 -> 6.22 | Helped |
| Use cross-entropy similarity loss only rather than the baseline training (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.34 -> 6.18 | Helped |
| Add intermediate alignment to cross-entropy similarity (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.18 -> 6.10 | Helped |
| Add intermediate alignment to L2 similarity (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.22 -> 6.17 | Helped |
| Add adversarial learning to cross-entropy similarity and intermediate alignment (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.10 -> 5.83 | Helped |
| Use L1 similarity with intermediate alignment and adversarial learning (ablation) | VGG-19 w/ BN · CIFAR-10 test | error rate (%): 6.34 -> 7.57 | Hurt |
Closest papers by meaning
- Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation (ICML 2022 · similarity 0.89)
- More or Less: When and How to Build Convolutional Neural Network Ensembles (ICLR 2021 · similarity 0.88)
- Ensemble Distribution Distillation via Flow Matching (ICML 2025 · similarity 0.86)
- Progressive Ensemble Distillation: Building Ensembles for Efficient Inference (NeurIPS 2023 · similarity 0.86)
- Deep Mutual Learning (CVPR 2018 · similarity 0.86)
- How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.