Research map / Learning theory and algorithms / Prediction and decision losses
Calibration by Distribution Matching: Trainable Kernel Calibration Metrics
Charles Thomas Marx, Sofian Zalouk, Stefano Ermon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MMD calibration regularizer to NLL training | regression MLP · crime | quantile calibration error: 0.220 -> 0.164 | Improved |
| add MMD calibration regularizer to NLL training | regression MLP · superconductivity | NLL: 3.375 -> 3.269 | Improved |
| add MMD calibration regularizer to NLL training | regression MLP · blog | NLL: 0.997 -> 0.957 | Improved |
| add MMD calibration regularizer to NLL training | regression MLP · crime | decision calibration error: 0.151 -> 0.042 | Improved |
| add post-hoc quantile recalibration to NLL-trained model | regression MLP · crime | NLL: -0.716 -> -0.387 | Hurt |
| add post-hoc quantile recalibration to NLL + MMD model | regression MLP · crime | quantile calibration error: 0.164 -> 0.105 | Improved |
| add post-hoc recalibration to NLL + MMD model | regression MLP · medical-expenditure | NLL: 1.532 -> 1.780 | Hurt |
| add MMD regularizer before post-hoc recalibration (reduces sharpness penalty) | regression MLP · fb-comment | NLL after post-hoc: 0.734 -> 0.639 | Improved |
| add post-hoc recalibration to NLL-trained model | regression MLP · blog | NLL: 0.997 -> 0.624 | Improved |
| replace cross-entropy with XE + MMD regularizer | classification MLP · breast-cancer | ECE: 0.194 -> 0.052 | Improved |
| replace cross-entropy with XE + MMD regularizer | classification MLP · heart-disease | accuracy: 55.904 -> 61.516 | Improved |
| use MMD instead of KDE regularizer | classification MLP · heart-disease | accuracy: 50.036 -> 61.516 | Improved |
Closest papers by meaning
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.87)
- Truthfulness of Calibration Measures (NeurIPS 2024 · similarity 0.87)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.87)
- Sampling-based Multi-dimensional Recalibration (ICML 2024 · similarity 0.86)
- Improving Model Probability Calibration by Integration of Large Data Sources with Biased Labels (AAAI 2025 · similarity 0.86)
- Combining Priors with Experience: Confidence Calibration Based on Binomial Process Modeling (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
