Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Epistemic Uncertainty Quantification To Improve Decisions From Black-Box Models
Sébastien Melo, Gaël Varoquaux, Marine Le Morvan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use binning-free honest-tree grouping loss estimator instead of binned estimator | Grouping loss estimation · Semi-synthetic Weather (TabReD) | grouping loss captured per sample size | Improved |
| increase LLM model size | LLM zero-shot confidence scores · Folktexts (27 LLMs, 5 ACS tasks) | normalized grouping loss: grouping loss decreases with model size | Improved |
| apply instruction tuning to base LLM | LLM zero-shot confidence scores · Folktexts (27 LLMs, 5 ACS tasks) | normalized grouping loss: lower (or equal) grouping loss than base counterparts | Improved |
| use epistemic risk cascade instead of using largest model | LLM pool · ACSIncome (Llama 3 instruct) | accuracy: +4% accuracy at about 30% of cost | Improved |
| use epistemic risk cascade instead of largest model | LLM pool · Folktexts 5 datasets, 120 pools | accuracy: 6% accuracy gain at 46% of the cost | Improved |
| add grouping risk (full epistemic risk) on top of calibration risk for deferral (ablation) | Cascade · ACSEmployment | accuracy: up to 15% | Improved |
| use epistemic risk cascade instead of predictive router | Cascade · Folktexts 5 datasets | accuracy and cost: strict Pareto optimality in 60% of cases | Improved |
| use full epistemic risk instead of calibration risk with more than 6000 training samples (ablation) | Cascade · All Folktexts datasets | accuracy or cost: consistently improves above 6,000 samples | Improved |
| use full epistemic risk with fewer than 6000 training samples (ablation) | Cascade · Folktexts datasets | accuracy: up to +12% with 2,000 samples on ACSEmployment | Mixed |
Closest papers by meaning
- To Believe or Not to Believe Your LLM: Iterative Prompting for Estimating Epistemic Uncertainty (NeurIPS 2024 · similarity 0.89)
- Epistemic Uncertainty Quantification For Pre-Trained Neural Networks (CVPR 2024 · similarity 0.89)
- Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning (NeurIPS 2026 · similarity 0.88)
- Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods? (ICML 2024 · similarity 0.88)
- Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.88)
- Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
