Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Single-model predictive uncertainty
Equips deep networks with uncertainty through SDEs, ensembles, bias reduction and cheap Bayesian conversion.
74 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Uncertainty and Gaussian processes, keyword cluster “uncertainty estimation · epistemic · quantification”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
14 papers in 2023–24 and 12 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 5.1% to 3.9%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (ICML 2016 · 2,628 citations)
- 2017 · Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2017 · 734 citations)
- 2018 · Evidential Deep Learning to Quantify Classification Uncertainty (NeurIPS 2018 · 543 citations)
- 2019 · Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift (NeurIPS 2019 · 924 citations)
- 2020 · Uncertainty Estimation Using a Single Deep Deterministic Neural Network (ICML 2020 · 275 citations)
- 2021 · Masksembles for Uncertainty Estimation (CVPR 2021 · 87 citations)
- 2022 · On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks (ICLR 2022 · 41 citations)
- 2023 · Gradient-Based Uncertainty Attribution for Explainable Bayesian Deep Learning (CVPR 2023 · 16 citations)
- 2024 · Deep Active Learning with Noise Stability (AAAI 2024 · 22 citations)
- 2025 · Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · 6 citations)
- 2026 · Possibilistic Predictive Uncertainty for Deep Learning (ICML 2026 · 0 citations)
Earliest papers here
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (ICML 2016 · 2,628 citations)
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2017 · 734 citations)
- Dropout Inference in Bayesian Neural Networks with Alpha-divergences (ICML 2017 · 105 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
4 papers, 2020–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 96 | Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors (ICML 2020 · 29 citations) |
CIFAR-10 · NLL (lower is better)
4 papers, 2019–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 0.163 | Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers (ICLR 2019 · 33 citations) |
| 2020 | 0.159 | Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors (ICML 2020 · 29 citations) |
What has been tried
67 papers here report 676 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 10%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add noise-deviation direction to magnitude-based selection | MNIST · classification performance | Helped | Deep Active Learning with Noise Stability (AAAI 2024 · 22 citations) | |
| Discard the loss term E | image classification experiments · performance | Hurt | Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models (CVPR 2024 · 16 citations) | |
| Use Laplace rather than Gaussian noise modeling | KITTI Eigen-split validation, corruption severity 5 · AUSE-REL | 0.071 -> 0.059 | Helped | Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regression (AAAI 2024 · 7 citations) |
| add gradient weighting to FLSD instead of weighting the loss | CIFAR10 · ECE | 1.26 -> 0.88 | Helped | Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · 6 citations) |
| Rescale covariance entries by layer using fine-tuning | GraphCast weather forecasting · log-likelihood | Helped | General Uncertainty Estimation with Delta Variances (AAAI 2025 · 1 citation) | |
| Use β=0.5 loss instead of standard DDPN loss | Length of Stay · CRPS | 0.361 -> 0.359 | Helped | Fully Heteroscedastic Count Regression with Deep Double Poisson Networks (ICML 2025 · 0 citations) |
| Set segmentation-loss weight λ_seg to 5 | ABIDE and OASIS3 · Dice | 0.759 ± 0.035 at λ_seg=2 -> 0.790 ± 0.019 at λ_seg=5; 0.734 ± 0.036 -> 0.772 ± 0.030 | Helped | Hierarchical Uncertainty Estimation for Learning-based Registration in Neuroimaging (ICLR 2025 · 0 citations) |
| Use temperature-scaled BIF rather than the standard temperature β=1 | CIFAR-10 human-noise mislabel detection · AUROC | Helped | Measuring Stochastic Data Complexity with Boltzmann Influence Functions (ICML 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 11 papers
- CIFAR-100 · 3 of 11 papers
Baselines
- MC-Dropout · 3 of 11 papers
- SNGP · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 26 papers; from OpenAlex and the papers' affiliations).
- University of Toronto · 2 papers
- Institut Polytechnique de Paris · 2 papers
- Google DeepMind · 2 papers
- Université Paris-Saclay · 2 papers
- City University of Hong Kong · 2 papers
- Monash University · 2 papers
Use it
Download BibTeX of 36 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Priors and functional variational uncertainty (112 papers)
- Epistemic versus aleatoric decomposition (75 papers)
- Bayesian decision-making under uncertainty (57 papers)
Elsewhere in the topic
- Nonparametric and PAC-Bayes Bayesian learning (130 papers)
- Simulation-based inference (118 papers)
- Variational Bayesian deep learning (105 papers)
- Approximate inference in Bayesian neural networks (93 papers)
- Sparse variational Gaussian processes (92 papers)
- PAC-Bayes and decoupled GP learning (83 papers)
- Bayesian view of meta-learning (63 papers)
- Infinite-width networks as Gaussian processes (49 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).