Research map / Architectures and efficiency / RNNs and normalization
Hyperparameter Ensembles for Robustness and Uncertainty Quantification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hyperparameter diversity via stratified hyper-deep ensemble instead of deep ensemble | WRN-28-10 · CIFAR-100 | NLL: 0.661 -> 0.652 | Helped |
| hyper-deep ensemble instead of deep ensemble | ResNet-20 · CIFAR-100 | NLL: 0.971 -> 0.925 | Helped |
| hyper-deep ensemble instead of deep ensemble | WRN-28-10 · CIFAR-10 | NLL: 0.125 -> 0.115 | Helped |
| hyper-deep ensemble instead of deep ensemble | ResNet-20 · CIFAR-10 | NLL: 0.193 -> 0.192 | No clear effect |
| hyper-batch ensemble instead of batch ensemble | ResNet-20 · CIFAR-100 | NLL: 1.235 -> 1.152 | Helped |
| hyper-batch ensemble instead of batch ensemble | WRN-28-10 · CIFAR-100 | NLL: 0.690 -> 0.678 | Helped |
| hyper-batch ensemble instead of batch ensemble | WRN-28-10 · CIFAR-100 | diversity: 0.761 -> 0.996 | Helped |
| stratify over initializations (hyper-deep) vs fixed init hyper ensemble (ablation) | MLP/LeNet · CIFAR-100 (LeNet) | NLL: 2.259 -> 2.211 | Helped |
| hyper-deep ens vs deep ens (MLP, accuracy) | MLP · CIFAR-100 (MLP) | accuracy: 0.289 -> 0.291 | No clear effect |
| hyper-batch ensemble vs batch ensemble on ECE | MLP/LeNet · CIFAR-100 | ECE | No clear effect |
| hyper-batch ensemble costs about 2x training time of batch ensemble | ResNet · CIFAR-100 | minutes/epoch: 1.10 -> 2.16 | Hurt |
Closest papers by meaning
- Neural Ensemble Search for Uncertainty Estimation and Dataset Shift (NeurIPS 2021 · similarity 0.90)
- More or Less: When and How to Build Convolutional Neural Network Ensembles (ICLR 2021 · similarity 0.88)
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2017 · similarity 0.88)
- Theoretical Limitations of Ensembles in the Age of Overparameterization (ICML 2025 · similarity 0.87)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.86)
- United We Stand: Using Epoch-Wise Agreement of Ensembles to Combat Overfit (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.