Research map / Architectures and efficiency / RNNs and normalization
Snapshot Ensembles: Train 1, Get M for Free
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cyclic learning-rate annealing and average snapshots saved at cycle minima | ResNet-50 · ImageNet validation | Top-1 error rate (%): 24.01 -> 23.33 | Helped |
| Use cyclic Snapshot Ensembling rather than a single model | DenseNet-100 · CIFAR-100 | error rate (%): 19.25 -> 17.41 | Helped |
| Add cyclic learning-rate restarts and ensemble snapshots | Wide-ResNet-32 · CIFAR-10 | error rate (%): 5.43 -> 4.41 | Helped |
| Replace non-cyclic training with cyclic learning-rate cycles (ablation) | Snapshot Ensemble · CIFAR-100 | error rate (%): 19.30 -> 17.41 | Helped |
| Replace cyclic training with a non-cyclic schedule (ablation) | Snapshot Ensemble · CIFAR-10 | error rate (%): 5.32 -> 5.49 | No clear effect |
| Reinitialize the model at the start of every cycle instead of continuing from previous cycle parameters (ablation) | Snapshot Ensemble · CIFAR-10 | error rate (%): 5.32 -> 6.66 | Hurt |
| Use a larger restart learning rate, α₀=0.2 rather than α₀=0.1 (ablation) | Snapshot Ensemble · CIFAR-100, DenseNet-100 | error rate (%): 18.12 -> 17.41 | Helped |
| Increase the number of cycles from 2 to 8 (ablation) | DenseNet-40 Snapshot Ensemble · CIFAR-100 | error rate (%): 22.92 -> 21.89 | Helped |
| Increase the number of cycles from 8 to 10 (ablation) | DenseNet-40 Snapshot Ensemble · CIFAR-100 | error rate (%): 21.89 -> 22.16 | Hurt |
| Use 3 cycles instead of 2 (ablation) | ResNet-50 Snapshot Ensemble · ImageNet validation | Top-1 error rate (%): 23.33 -> 23.96 | Hurt |
| Average predictions from multiple snapshots instead of using one snapshot (ablation) | Snapshot Ensemble · CIFAR-100 | error rate (%) | Helped |
| Use later, converged snapshots rather than early snapshots in the ensemble (ablation) | Snapshot Ensemble · CIFAR-10 and CIFAR-100 | error rate (%) | Helped |
Closest papers by meaning
- To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning (NeurIPS 2023 · similarity 0.89)
- Collegial Ensembles (NeurIPS 2020 · similarity 0.88)
- Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity (ICLR 2022 · similarity 0.88)
- Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems (NeurIPS 2024 · similarity 0.87)
- Theoretical Limitations of Ensembles in the Age of Overparameterization (ICML 2025 · similarity 0.85)
- Rethinking LLM Ensembling from the Perspective of Mixture Models (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.