Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
How Good is the Bayes Posterior in Deep Neural Networks Really?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Cool the posterior below temperature 1 | Bayes ensemble · CIFAR-10 and IMDB | test predictive performance | Helped |
| Use temperature 10^-4 rather than temperature 1 | ResNet-20 Bayes ensemble · CIFAR-10 | test performance | Helped |
| Reduce posterior temperature below 1 | CNN-LSTM Bayes ensemble · IMDB | test predictive performance | Helped |
| Use smaller minibatches | SG-MCMC ensembles · CIFAR-10 and IMDB | cross-entropy | Helped |
| Increase ensemble size toward infinity at temperature 1 | ResNet-20 SG-MCMC ensemble · CIFAR-10 | test cross-entropy: 0.341 | No clear effect |
| Add layerwise preconditioning (ablation) | SG-MCMC with cyclical stepping · IMDB | accuracy: 51.3±0.65 -> 81.95±0.22 | Helped |
| Add cyclical time stepping (ablation) | SG-MCMC with layerwise preconditioning · IMDB | accuracy: 74.5±0.49 -> 81.95±0.22 | Helped |
| Add layerwise preconditioning and cyclical time stepping (ablation) | SG-MCMC · IMDB | cross-entropy: 0.698±0.0013 -> 0.425±0.0032 | Helped |
| Add cyclical time stepping (ablation) | SG-MCMC without preconditioning · CIFAR-10 | cross-entropy: 0.507±0.0080 -> 0.3500±0.00064 | Helped |
| Add layerwise preconditioning (ablation) | SG-MCMC without cyclical stepping · CIFAR-10 | cross-entropy: 0.507±0.0080 -> 0.3808±0.00064 | Helped |
| Use layerwise preconditioning and cyclical stepping together (ablation) | SG-MCMC · CNN-LSTM IMDB | simulation accuracy | Helped |
| Use cyclical stepping without preconditioning (ablation) | SG-MCMC · ResNet-20 CIFAR-10 | simulation accuracy | Helped |
Closest papers by meaning
- A statistical theory of cold posteriors in deep neural networks (ICLR 2021 · similarity 0.91)
- A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors (ICLR 2024 · similarity 0.89)
- Do Bayesian Neural Networks Actually Behave Like Bayesian Models? (ICML 2025 · similarity 0.88)
- Scalable Bayesian Learning with posteriors (ICLR 2025 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.87)
- Martingale Posterior Neural Networks for Fast Sequential Decision Making (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.