Research map / Architectures and efficiency / RNNs and normalization
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling
Gregory Benton, Wesley Maddox, Sanae Lotfi, Andrew Gordon Gordon Wilson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a 2-simplex at each of three independently trained ensemble components | VGG-16 deep ensemble · CIFAR-10 | test error: 6.2% -> 5.7% | Improved |
| Use ESPRO simplexes instead of independent models at a fixed training budget | VGG-16 ensemble · CIFAR-10 and CIFAR-100 | test error | Improved |
| Use ESPRO rather than a standard deep ensemble at a fixed ensemble size | VGG-16 ensemble · CIFAR-10 and CIFAR-100 | test error | Improved |
| Increase simplex order within ESPRO (ablation) | VGG-16 ensemble · CIFAR-10 and CIFAR-100 | test error | Improved |
| Increase the number of independent ensemble components (ablation) | ESPRO and deep ensembles · CIFAR-10 | test error | Improved |
| Use a SPRO simplex rather than a deep ensemble for transformer models | Image transformer · CIFAR-100 | test error | No clear effect |
| Use a four-dimensional ESPRO simplex instead of a deep ensemble (ablation) | Image transformer · CIFAR-100 | test error | Improved |
| Use a four-dimensional ESPRO simplex instead of a deep ensemble (ablation) | Image transformer · CIFAR-100 | NLL | Hurt |
| Ensemble over a mode-connecting simplex complex rather than individual models (ablation) | SPRO mode-connecting complex · CIFAR-10 and CIFAR-100 | test error | No clear effect |
| Add a connector after the first ten connectors (ablation) | SPRO mode-connecting complex · CIFAR-10, VGG-16 | simplex volume: 10^5 -> approximately 10^{-4} | Hurt |
| Increase simplex sample count beyond approximately 25 (ablation) | SPRO ensemble · CIFAR-100 | test error | No clear effect |
| Apply temperature scaling to predictions under Gaussian noise corruption | ESPRO and comparison methods · CIFAR-10 Gaussian noise corruption | NLL | Improved |
Closest papers by meaning
- Simplicity Bias via Global Convergence of Sharpness Minimization (ICML 2024 · similarity 0.84)
- Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.84)
- On Linear Mode Connectivity of Mixture-of-Experts Architectures (NeurIPS 2025 · similarity 0.83)
- The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws (NeurIPS 2026 · similarity 0.83)
- Input Space Mode Connectivity in Deep Neural Networks (ICLR 2025 · similarity 0.83)
- Understanding Mode Connectivity via Parameter Space Symmetry (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
