Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add KL regularisation and Langevin noise to deep ensembles | Deep ensembles · UCI regression benchmarks | test negative log likelihood | Mixed |
| Add MMD-based repulsive interactions and KL regularisation | Deep Langevin ensembles · UCI regression benchmarks | test negative log likelihood | Mixed |
| Use a finite Gaussian variational family instead of an infinite-dimensional particle distribution | GVI · Multimodal toy loss | approximation quality to the global minimiser | Hurt |
| Add regularisation and Langevin noise | Deep ensembles · Multimodal toy loss | particle spread around local minima | Helped |
| Add MMD repulsion to Langevin particles | Deep Langevin ensembles · Multimodal toy loss | global-minimiser convergence | Helped |
| Use unregularised gradient descent from random initialisations rather than a regularised objective | Deep ensembles · Loss landscape with multiple local minima | convergence to the global minimiser | Hurt |
| Use few particles relative to the number of local minima | Deep ensembles, DLE, and DRLE · Toy loss with 2000 local minima | approximation quality to the global minimiser: 2000; 20; 16/20 | Hurt |
Closest papers by meaning
- Variational inference via Wasserstein gradient flows (NeurIPS 2022 · similarity 0.89)
- Good Initializations of Variational Bayes for Deep Models (ICML 2019 · similarity 0.89)
- Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift (NeurIPS 2023 · similarity 0.88)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.88)
- Variational Bayesian Monte Carlo (NeurIPS 2018 · similarity 0.88)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.