Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use deep ensembles instead of the other evaluated uncertainty methods | ResNet-20 on CIFAR-10 · CIFAR-10 corruptions | Brier score: 0.425 -> 0.299 | Helped |
| Use deep ensembles instead of the other evaluated uncertainty methods | ResNet-50 on ImageNet · ImageNet corruptions | Brier score: 0.733 -> 0.667 | Helped |
| Use deep ensembles instead of the other evaluated uncertainty methods | MLP on Criteo · Criteo categorical-feature shift | Brier score: 0.385 -> 0.366 | Helped |
| Apply temperature scaling on the i.i.d. validation set | Vanilla MLP · Criteo categorical-feature shift | Brier score: 0.385 -> 0.391 | Hurt |
| Use full-network dropout rather than last-layer-only dropout | ResNet-20 on CIFAR-10 · CIFAR-10 corruptions | Brier score: 0.416 -> 0.349 | Helped |
| Use full-network dropout rather than last-layer-only dropout | LSTM on 20 Newsgroups · Shifted and OOD text | confidence versus accuracy | Helped |
| Increase the number of prediction samples for Dropout or SVI (ablation) | CIFAR-10 models · CIFAR-10 corruptions | Brier score | Helped |
| Increase the number of prediction samples for last-layer variants (ablation) | Last-layer Dropout and LL-SVI · CIFAR-10 corruptions | Brier score | No clear effect |
| Increase ensemble size, especially up to 5 models (ablation) | CIFAR-10 and ImageNet models · Image corruptions | Brier score | Helped |
| Double the number of filters (ablation) | Vanilla and Dropout ResNet models · CIFAR-10 medium- to high-shift corruptions | accuracy and calibration | No clear effect |
| Add Bayesian parameter uncertainty using SVI | LeNet on MNIST · MNIST shifted images | Brier score | Helped |
| Add Bayesian parameter uncertainty using SVI | Models on Criteo · Criteo categorical-feature shift | Brier score: 0.385 -> 0.512 | Hurt |
Closest papers by meaning
- Accurate Uncertainties for Deep Learning Using Calibrated Regression (ICML 2018 · similarity 0.90)
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.87)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.87)
- Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows (NeurIPS 2025 · similarity 0.86)
- Truthful Calibration Errors for Multi-Class Prediction (NeurIPS 2026 · similarity 0.86)
- Provable Uncertainty Decomposition via Higher-Order Calibration (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.