AI Paper Atlas

Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinations

Vihari Piratla, Soumen Chakrabarti, Sunita Sarawagi

NeurIPS 2021 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Structured Gaussian process models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add Dirichlet supervision of Beta scale to BetaGP (ablation)BetaGP · AC-COCOS10K, AC-COCOS, MF-IMDB, MF-CelebAmacro MSE: 7.1 / 0.3 -> 5.3 / 0.2; 3.3 / 0.2 -> 2.8 / 0.0; 2.2 / 0.2 -> 1.4 / 0.1; 4.6 / 0.8 -> 4.1 / 0.1Improved
Add sparse-arm observation pooling to BetaGP-SL (ablation)BetaGP-SL · AC-COCOS10K, AC-COCOS, MF-IMDB, MF-CelebAworst MSE: 11.7 / 1.7 -> 10.4 / 1.5; 6.8 / 0.5 -> 5.7 / 0.5; 22.6 / 1.4 -> 23.3 / 2.3; 4.4 / 0.6 -> 3.9 / 0.7Mixed
Use Beta mean and scale parameters rather than standard (alpha, beta) parameters (ablation)BetaGP · MF-CelebA and AC-COCOSmacro MSE: 6.2 / 0.1 -> 4.6 / 0.8; 4.8 / 0.1 -> 3.3 / 0.2Improved
Smooth accuracy across related arms using a GP (ablation)Beta-I · MF-CelebAworst MSE: 8.4 / 0.7 -> 7.6 / 1.3Improved
Calibrate attribute probabilities with temperature scaling and a joint potential (ablation)Raw attribute predictions · Four classification tasksmacro MSEImproved
Add temperature calibration without the joint-potential term (ablation)Raw attribute predictions · Four classification tasksmacro MSEImproved
Use GP-smoothed mean and scale to model arm-specific uncertainty instead of Bernoulli GP classification (ablation)BernGP · Four classification tasksestimation qualityImproved
Use BetaGP for variance-based exploration without scale supervision or pooling (ablation)BetaGP · Four classification tasksmacro MSE during explorationHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy