AI Paper Atlas

Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Learning Prediction Intervals for Model Performance

Benjamin Elder, Matthew Arnold, Anupama Murthi, Jiří Navrátil

AAAI 2021 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Regression uncertainty quantification · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use a transfer-trained uncertainty model instead of the strongest calibrated baselinePerformance prediction intervals with the Meta-model predictor · linear-skew train → linear-skew evalaverage cost: 1.77 -> 1.33Improved
Use a transfer-trained uncertainty model instead of the strongest calibrated baselinePerformance prediction intervals with the Meta-model predictor · linear-skew train → nearest-neighbors evalaverage cost: 1.15 -> 0.97Improved
Add Distance features to Prediction features (ablation)Uncertainty model · Feature ablation; meta-model predictoraverage cost (α=0.9; normalized): 1.33 -> 1.07Improved
Add Internal features to Prediction features (ablation)Uncertainty model · Feature ablation; meta-model predictoraverage cost (α=0.9; normalized): 1.33 -> 1.05Improved
Add Drift features to the feature set (ablation)Uncertainty model · Feature ablation; meta-model predictoraverage cost (α=0.9; normalized): 1.16 -> 1.0Improved
Use all feature types instead of Prediction features only (ablation)Uncertainty model · Feature ablation; meta-model predictoraverage cost (α=0.9; normalized): 1.33 -> 1.0Improved
Use all feature types instead of Noise features only (ablation)Uncertainty model · Feature ablation; meta-model predictoraverage cost (α=0.9; normalized): 1.32 -> 1.0Improved
Replace the Meta-model predictor with CrossvalPerformance prediction · linear-skew scenariosMAE: 3.84 -> 4.04Hurt
Replace the Meta-model predictor with DropoutPerformance prediction · linear-skew scenariosMAE: 3.84 -> 4.70Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy