Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified framework decomposing pointwise risk for strictly proper scoring rules into aleatoric (Bayes risk), epistemic (excess risk) and total components, estimated via Bayesian/deep-ensemble approximations, and evaluates the resulting measures for OOD and misclassification detection.
Problem
Predictive uncertainty measures in the literature are ad hoc and their relationships are unclear; there is no unified formal definition of aleatoric and epistemic uncertainty.
Approach
Decompose pointwise risk under strictly proper scoring rules (generated by a convex G) into Bayes (aleatoric), excess (epistemic) and total risk, then estimate each with Bayesian approximations (expectation over posterior, central label, central prediction), yielding a family of measures that recover known ones such as mutual information, EPKL, and energy-based scores.
Key results
- For OOD detection with CIFAR10 in-distribution, Log-score-based measures generally outperform Brier and Spherical scores (e.g., Bayes risk Log score reaches 91.36 AUROC on CIFAR100 and 96.01 on SVHN).
- For soft-OOD (ImageNet-O with TinyImageNet), excess risk measures are competitive, but for hard-OOD (ImageNet-A, ImageNet-R) Bayes risk typically outperforms excess risk.
- For misclassification detection on noisy CIFAR datasets, Bayes and total risks outperform excess risk, with gaps widening as label noise increases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bayes risk instead of Excess risk for hard-OOD detection (ImageNet-A etc.) | Bayesian risk measures on TinyImageNet · TinyImageNet OOD | AUROC: Bayes (2) 83.76 vs Excess (1,2) 79.09 on ImageNet-A | Helped |
| Excess risk (EPKL, Log score) used for soft-OOD detection instead of Bayes risk | Excess risk (1,1) Log score · CIFAR10 OOD (CIFAR100, SVHN) | AUROC: 90.17 vs 91.36 (CIFAR100); 94.08 vs 96.01 (SVHN) | Hurt |
| Adding Total risk instead of Excess/Bayes risk for misclassification detection | Total risk (3,1) Log score · CIFAR10/CIFAR100-N misclassification | AUROC: 94.74 vs 94.1 (EPKL) on CIFAR10; 83.18 vs 74.52 on CIFAR100-N | Helped |
| Log score as loss/plug-in G instead of Brier or Spherical scores (ablation) | Risk measures for OOD detection · CIFAR10 OOD | AUROC: Log Bayes (1) 91.36 vs Brier 90.44 and Spherical 90.46 on CIFAR100 | Helped |
| Energy from expected logits E(x;E f) versus expected per-member energy as OOD score (ablation) | Energy-based measure on TinyImageNet · TinyImageNet OOD ImageNet-A | AUROC: 83.96 vs 82.99 | Helped |
| Excess risk MRBI (3,1) vs energy estimates for OOD | Excess risk (3,1) Log score · TinyImageNet OOD ImageNet-A | AUROC: 77.56 vs 83.96 for E(x;E f) | Hurt |
| Bayes risk approximation (1) vs (2) for aleatoric measure on CIFAR10 misclassification (ablation) | Bayes risk Log score · CIFAR10 misclassification | AUROC: 94.39 (1) vs 94.7 (2) vs 94.76 (3) | Mixed |
Surprising findings
- For hard OOD data, the aleatoric Bayes risk outperforms epistemic excess risk measures, contrary to the common expectation that epistemic uncertainty should handle OOD detection.
- Excess risk measures such as Bregman Information are not clearly the best epistemic measure; no single approximation of excess risk is uniformly best.
Limitations
- Assumes the true conditional eta(x) is valid (a K-length probability vector) even for out-of-distribution inputs, which is vague in practice.
- Excess risk estimates are neither upper nor lower bounds on the true excess risk, so the best approximation depends on the unknown regime of total-risk estimation.
- Experiments are limited to deep ensembles of ResNet18 on image datasets; the framework is not compared against state-of-the-art uncertainty methods.
Experimental setup
Datasets: CIFAR10 CIFAR100 TinyImageNet CIFAR10C SVHN ImageNet-A ImageNet-R ImageNet-O
Benchmarks: CIFAR10C SVHN ImageNet-A ImageNet-R ImageNet-O CIFAR10-N CIFAR100-N
Metrics: AUROC
Baselines: mutual information expected pairwise KL energy score maximum probability predictive variance
Builds on: deep ensembles mutual information expected pairwise KL DEUP energy-based models
Closest papers by meaning
- Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks (NeurIPS 2024 · similarity 0.89)
- Epistemic Uncertainty for Generated Image Detection (NeurIPS 2025 · similarity 0.85)
- Second-Order Uncertainty Quantification: A Distance-Based Approach (ICML 2024 · similarity 0.85)
- Provable Uncertainty Decomposition via Higher-Order Calibration (ICLR 2025 · similarity 0.85)
- Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.85)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.