Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Quantifying Uncertainty in the Presence of Distribution Shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VIDS, a Bayesian uncertainty method whose prior is conditioned on test covariates, approximated with amortized variational inference and trained across bootstrap-based synthetic covariate-shift environments.
Problem
Neural network uncertainty estimates become unreliable under covariate shift between training and test inputs, since the classical Bayesian posterior predictive ignores test covariates.
Approach
Makes the prior over network parameters an energy-based function of both training covariates and the test covariate, so the posterior adapts to how new covariates change predictive performance; amortizes the variational posterior over test covariates and training sets, and trains it on bootstrap-subsampled synthetic environments with a cross-environment variance penalty.
Key results
- On synthetic linear regression, VIDS achieves RMSE 0.068 versus 0.110 (DUE), 0.488 (SNGP), and 0.081 (DUL).
- On the synthetic logistic task, VIDS achieves accuracy 0.900 and ACE 0.011, the lowest calibration error among compared methods.
- VIDS reports the highest accuracy in all 6 classification experiments (CIFAR-10-C and Celeb-A) and the lowest or comparable RMSE on the three UCI regression datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VIDS instead of DUL | Synthetic linear regression · Synthetic linear regression | RMSE: 0.081 (0.010) -> 0.068 (0.002) | Helped |
| Use VIDS instead of DUL | Synthetic logistic regression · Synthetic logistic regression | Accuracy: 0.900 (0.008) -> 0.900 (0.009) | No clear effect |
| Use VIDS instead of DUL | Synthetic logistic regression · Synthetic logistic regression | ACE: 0.018 (0.000) -> 0.011 (0.003) | Helped |
| Use VIDS instead of competing methods | VIDS · Corrupted CIFAR-10 and Celeb-A classification experiments | Accuracy | Helped |
| Use VIDS instead of competing methods | VIDS · Boston, Concrete, and Wine regression | RMSE | Helped |
Surprising findings
- DUL matches VIDS accuracy on the synthetic logistic task (0.900) despite overestimating variance.
- VIDS is slower than baselines on small models such as linear and logistic regression because of per-environment optimization.
Limitations
- Computationally more expensive for small models due to per-environment optimization.
- Relies on pre-trained embedding network and optimizes only the prediction layer.
- Requires an integration over the outcome space for the prior, approximated by Monte Carlo for continuous outputs.
Experimental setup
Datasets: CIFAR-10 CIFAR-10-C CelebA UCI Boston UCI Concrete UCI Wine
Benchmarks: CIFAR-10-C CelebA UCI Boston UCI Concrete UCI Wine synthetic heteroscedastic linear regression synthetic logistic regression with missing data
Metrics: accuracy RMSE expected calibration error (ACE) runtime
Baselines: SNGP DUE DUL
Builds on: Bayesian neural networks amortized variational inference deep sets reparametrization trick deterministic uncertainty estimation
Closest papers by meaning
- Dangers of Bayesian Model Averaging under Covariate Shift (NeurIPS 2021 · similarity 0.91)
- Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift (ICML 2020 · similarity 0.90)
- Not all distributional shifts are equal: Fine-grained robust conformal inference (ICML 2024 · similarity 0.90)
- Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows (NeurIPS 2025 · similarity 0.87)
- Neural Conditional Probability for Uncertainty Quantification (NeurIPS 2024 · similarity 0.87)
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.