Research map / Learning theory and algorithms / Prediction and decision losses
Measuring Uncertainty Calibration
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian, Juan Elenter, Francesco Tonolini, David Gustafsson, Eva Garcia-Martin, Carmen Barcena Gonzalez, Raphaëlle Bertrand-Lalo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| perturb classifier outputs by h=2^-6 | BERT and ViT classifiers · IMDB, Spam Detection, CIFAR | AUROC | No clear effect |
| use kernel (NW) smoother under bounded derivatives instead of Lipschitz bucketing | NW estimator · Synthetic and real datasets | bound tightness | Improved |
| use TV denoising under bounded variation | TV estimator · Synthetic datasets | empirical rate: [-0.423, -0.164] vs theory -1/4 | Mixed |
| rely on ECE heuristic instead of certified bounds | ECE · Synthetic calibration function 4 | gap to true calibration error | Hurt |
| rely on ECE heuristic instead of certified bounds | ECE · Synthetic calibration functions 1-3 | gap to true calibration error | No clear effect |
Closest papers by meaning
- The Minimax Rate of Second-Order Calibration (NeurIPS 2026 · similarity 0.89)
- Confidence Calibration of Classifiers with Many Classes (NeurIPS 2024 · similarity 0.87)
- Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling (NeurIPS 2025 · similarity 0.87)
- Truthful Calibration Errors for Multi-Class Prediction (NeurIPS 2026 · similarity 0.86)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.86)
- Towards Certification of Uncertainty Calibration under Adversarial Attacks (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
