Research map / Learning theory and algorithms / Prediction and decision losses
Reliable Decisions with Threshold Calibration
Roshni Sahoo, Shengjia Zhao, Alyssa Chen, Stefano Ermon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add average calibration to the uncalibrated Gaussian forecaster | Gaussian forecaster · UCI Protein | reliability gap | Mixed |
| Add threshold calibration to the uncalibrated Gaussian forecaster | Gaussian forecaster · MIMIC-III and UCI Protein, across decision thresholds | reliability gap | Improved |
| Add distribution calibration to the uncalibrated Gaussian forecaster | Gaussian forecaster · MIMIC-III and UCI Protein, across decision thresholds | reliability gap | Improved |
| Add average calibration to the Gaussian forecaster | Gaussian forecaster · UCI Protein, across cost ratios | reliability gap | No clear effect |
| Add threshold calibration to the Gaussian forecaster | Gaussian forecaster · MIMIC-III and UCI Protein, across cost ratios | reliability gap | Improved |
| Add any recalibration (average, threshold, distribution) to the Gaussian forecaster | Gaussian forecaster · MIMIC-III and UCI Protein | decision loss | No clear effect |
| Add threshold calibration to the Gaussian-Laplace mixture forecaster | Gaussian-Laplace forecaster · MIMIC-III and UCI Protein | reliability gap | Improved |
| Add distribution calibration to the Gaussian-Laplace mixture forecaster | Gaussian-Laplace forecaster · MIMIC-III and UCI Protein | reliability gap | Hurt |
| Add distribution calibration to the Gaussian-Laplace mixture forecaster | Gaussian-Laplace forecaster · MIMIC-III and UCI Protein | decision loss | Hurt |
| Add average or threshold calibration to the Gaussian-Laplace mixture forecaster | Gaussian-Laplace forecaster · MIMIC-III and UCI Protein | decision loss | No clear effect |
| Use a more flexible Gaussian-Laplace mixture instead of a Gaussian forecaster (uncalibrated) | Uncalibrated forecaster · MIMIC-III and UCI Protein | reliability gap and decision loss | Improved |
Closest papers by meaning
- Truthfulness of Calibration Measures (NeurIPS 2024 · similarity 0.87)
- Algorithms with Calibrated Machine Learning Predictions (ICML 2025 · similarity 0.86)
- Testable and Actionable Calibration for Full Swap Regret (NeurIPS 2026 · similarity 0.85)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.84)
- Simultaneous Swap Regret Minimization via KL-Calibration (NeurIPS 2025 · similarity 0.83)
- Optimal Multiclass U-Calibration Error and Beyond (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
