Research map / Architectures and efficiency / RNNs and normalization
BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace a single ResNet32x4 with a size-4 BatchEnsemble | ResNet32x4 · CIFAR-10 validation | accuracy: 95.31 -> 95.94 | Helped |
| Replace a single ResNet32x4 with a size-4 BatchEnsemble | ResNet32x4 · CIFAR-100 validation | accuracy: 78.32 -> 80.32 | Helped |
| Use BatchEnsemble instead of naive ensembling | ResNet32x4 · CIFAR-10 validation | accuracy: 96.30 -> 95.94 | Hurt |
| Use BatchEnsemble instead of naive ensembling | ResNet32x4 · CIFAR-100 validation | accuracy: 81.02 -> 80.32 | Hurt |
| Replace a single Transformer big with BatchEnsemble | Transformer big · WMT16 EN-DE Newstest2013 | perplexity: 4.30 -> 4.26 | Helped |
| Replace a single Transformer big with BatchEnsemble | Transformer big · WMT14 EN-FR Newstest2013 | perplexity: 2.76 -> 2.74 | Helped |
| Use BatchEnsemble instead of dropout sampling | Transformer big · WMT16 EN-DE Newstest2013 | perplexity: 4.30 -> 4.26 | Helped |
| Use BatchEnsemble instead of dropout sampling | Transformer big · WMT14 EN-FR Newstest2013 | perplexity: 2.77 -> 2.74 | Helped |
| Use BatchEnsemble in place of PNN | LeNet lifelong learning · Split-CIFAR100 | computational cost relative to vanilla: 1.12 -> 1.11 | No clear effect |
| Use BatchEnsemble in place of PNN | LeNet lifelong learning · Split-CIFAR100 | memory cost relative to vanilla: 4.16 -> 1.10 | Helped |
| Use BatchEnsemble with ensemble size 8 instead of dropout | Contextual bandit uncertainty model · Contextual bandits | M.VALUE: 36.73 -> 33.95 | Helped |
| Use BatchEnsemble rather than a single model | ResNet32 · CIFAR-10 with 50% training data | accuracy: 92.20 -> 92.93 | Helped |
Closest papers by meaning
- Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks (AAAI 2022 · similarity 0.89)
- Snapshot Ensembles: Train 1, Get M for Free (ICLR 2017 · similarity 0.87)
- Hyperparameter Ensembles for Robustness and Uncertainty Quantification (NeurIPS 2020 · similarity 0.87)
- Efficient Lifelong Model Evaluation in an Era of Rapid Progress (NeurIPS 2024 · similarity 0.85)
- Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems (NeurIPS 2024 · similarity 0.85)
- Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.