Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Approximate inference in Bayesian neural networks
Examines how well approximate posteriors and cold posteriors behave in Bayesian neural networks.
93 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Uncertainty and Gaussian processes, keyword cluster “bayesian inference · posterior · priors”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 2 tracked benchmarks.
24 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 8.8% to 4.9%.
Open questions the data points to
- CIFAR-100 (accuracy): best went from 84.52 (up to 2024) to 87.63 in 2025 — Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · 6 citations)
What looks settled
- CIFAR-10 (accuracy): best up to 2024 97.11, best in 2025–26 97.57 from 2 papers, less than 1% better.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A Theoretically Grounded Application of Dropout in Recurrent Neural Networks (NeurIPS 2016 · 887 citations)
- 2017 · Learning Bayesian Networks with Incomplete Data by Augmentation (AAAI 2017 · 8 citations)
- 2018 · Adversarial Distillation of Bayesian Neural Network Posteriors (ICML 2018 · 30 citations)
- 2019 · Deterministic Variational Inference for Robust Bayesian Neural Networks (ICLR 2019 · 59 citations)
- 2020 · Capsule Routing via Variational Bayes (AAAI 2020 · 66 citations)
- 2021 · What Are Bayesian Neural Network Posteriors Really Like? (ICML 2021 · 99 citations)
- 2022 · Transformers Can Do Bayesian Inference (ICLR 2022 · 25 citations)
- 2023 · Kalman Bayesian Neural Networks for Closed-Form Online Learning (AAAI 2023 · 14 citations)
- 2024 · Full Bayesian Significance Testing for Neural Networks (AAAI 2024 · 5 citations)
- 2025 · Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · 6 citations)
- 2026 · Multi-Task Bayesian In-Context Learning (ICML 2026 · 0 citations)
Earliest papers here
- A Theoretically Grounded Application of Dropout in Recurrent Neural Networks (NeurIPS 2016 · 887 citations)
- Assumed Density Filtering Methods for Learning Bayesian Neural Networks (AAAI 2016 · 37 citations)
- Natural-Parameter Networks: A Class of Probabilistic Neural Networks (NeurIPS 2016 · 35 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
7 papers, 2020–2025 · small gains only
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 88.2 | How Good is the Bayes Posterior in Deep Neural Networks Really? (ICML 2020 · 32 citations) |
| 2021 | 89.64 | What Are Bayesian Neural Network Posteriors Really Like? (ICML 2021 · 99 citations) |
| 2023 | 97.11 | Flat Seeking Bayesian Neural Networks (NeurIPS 2023 · 1 citation) |
| 2025 | 97.57 | Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · 6 citations) |
CIFAR-100 · accuracy (higher is better)
5 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 84.52 | Flat Seeking Bayesian Neural Networks (NeurIPS 2023 · 1 citation) |
| 2025 | 87.63 | Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · 6 citations) |
What has been tried
82 papers here report 838 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 12%, hurt 16%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| increase low-rank adapter rank from 2 toward 16 | CAMELYON17 · accuracy | 95.08% -> 95.21% | Helped | Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · 6 citations) |
| Use SNR of absolute weights for removal instead of the other tested removal criteria | CIFAR-100 · accuracy | Helped | Training Bayesian Neural Networks with Sparse Subspace Variational Inference (ICLR 2024 · 2 citations) | |
| add 10 selected feature interactions | MIMIC-III ICU Mortality · AUROC | 79.6 ± 0.01 -> 80.2 ± 0.10 | Helped | Improving Neural Additive Models with Bayesian Principles (ICML 2024 · 2 citations) |
| replace the log-uniform prior with tVAMP | Sinusoidal time series · NLL | -0.092 -> -0.166 | Helped | BARNN: A Bayesian Autoregressive and Recurrent Neural Network (ICML 2025 · 1 citation) |
| Include upward segments in the prior | IC and ColPret · RMSLE | IC: 0.032 -> 0.028; ColPret: 0.032 -> 0.027 | Helped | Bayesian Neural Scaling Law Extrapolation with Prior-Data Fitted Networks (ICML 2025 · 0 citations) |
| use HiLoFi, with full-rank last-layer covariance and low-rank hidden-layer covariance, instead of LoLoFi | MNIST contextual bandit · cumulative reward (pBayes) | 30,038.5 -> 31,167.7 | Helped | Martingale Posterior Neural Networks for Fast Sequential Decision Making (NeurIPS 2025 · 0 citations) |
| use a linear approximation for the utility function | cross-stitch and chain instances · circuit size | cross-stitch: 327 -> 352, 815 -> 554, 2 220 -> 1 244; chain: 259 -> 205, 632 -> 461, 1 851 -> 648 | Mixed | Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation (AAAI 2024 · 0 citations) |
| Average weights of NPTL posterior samples | Image classification (Avg.) · ACC | 0.854 -> 0.848 | Hurt | Enhancing Transfer Learning with Flexible Nonparametric Posterior Sampling (ICLR 2024 · 0 citations) |
| Increase DEI-MCMC sampling from 10 to 1000 samples | airfoil, bikesharing, concrete, energy, protein, yacht · LPPD | 0.50±0.01 -> 0.58±0.03 (airfoil); 0.60±0.02 -> 0.63±0.02 (bikesharing); 0.05±0.27 -> 0.23±0.12 (concrete); 1.91±0.28 -> | Helped | Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks? (ICML 2024 · 0 citations) |
| Use memory-efficient loss computation instead of naive training computation | CIFAR10, 10 ipc · training memory | 542.9 -> 272.9 MB | Helped | Variational Bayesian Pseudo-Coreset (ICLR 2025) |
| Apply non-negativity constraints to decision weights and input features | CIFAR-100 · accuracy | 74.62 ± 0.23 -> 76.73 ± 0.16 | Helped | Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer (ICLR 2025) |
What reviewers will expect
Datasets and baselines named in the 10 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 10 papers
- CIFAR-100 · 4 of 10 papers
- MNIST · 2 of 10 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 39 papers; from OpenAlex and the papers' affiliations).
- Korea Advanced Institute of Science and Technology · 5 papers
- Albert-Ludwigs-Universität Freiburg · 3 papers
- Ludwig-Maximilians-Universität München · 2 papers
- LMU Munich · 2 papers
- ELLIS Institute Tübingen · 2 papers
- University of Freiburg · 2 papers
- Prior Labs · 2 papers
- Universität Freiburg · 2 papers
- University of Oxford · 2 papers
- University of Cambridge · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Nonparametric and PAC-Bayes Bayesian learning (130 papers)
- Simulation-based inference (118 papers)
- Variational Bayesian deep learning (105 papers)
- Bayesian view of meta-learning (63 papers)
Elsewhere in the topic
- Priors and functional variational uncertainty (112 papers)
- Sparse variational Gaussian processes (92 papers)
- PAC-Bayes and decoupled GP learning (83 papers)
- Epistemic versus aleatoric decomposition (75 papers)
- Single-model predictive uncertainty (74 papers)
- Bayesian decision-making under uncertainty (57 papers)
- Infinite-width networks as Gaussian processes (49 papers)
- Regression uncertainty quantification (43 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).