Research map / Learning theory and algorithms / Prediction and decision losses
T-Cal: An Optimal Test for the Calibration of Predictive Models
Donghwan Lee, Xinmeng Huang, Hamed Hassani, Edgar Dobriban
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use debiased l2-ECE statistic instead of plug-in biased l2 (ablation) | T-Cal · synthetic mis-calibration (P_1,m) | type II error | Improved |
| use l2-ECE statistic instead of plug-in l1-ECE (ablation) | T-Cal · synthetic mis-calibration (P_1,m) | type II error | Improved |
| T-Cal versus classical logistic-refit and consistency-resampling tests | T-Cal · synthetic mis-calibration (P_1,m), n=10,000 | true detection rate | Improved |
| increase sample size n from 2,000 to 10,000 | T-Cal · synthetic mis-calibration (P_1,m) | type II error: n = 2,000, 5,000, 10,000 | Improved |
| apply Platt scaling post-hoc to CIFAR-100 ResNet 56 | ResNet 56 · CIFAR-100 | empirical l1-ECE / T-Cal verdict: 15.2% -> 1.84% | Improved |
| compare lower l1-ECE with test verdict | ResNet 56 · CIFAR-100 | T-Cal verdict: 1.84% accepted vs 1.57% rejected | Mixed |
| apply polynomial scaling to ImageNet DenseNet 161 | DenseNet 161 · ImageNet | empirical l1-ECE / T-Cal verdict: 5.67% -> 0.62% | Improved |
| apply Platt scaling to CIFAR-10 DenseNet 121 | DenseNet 121 · CIFAR-10 | empirical l1-ECE: 2.02% -> 2.32% | Hurt |
| apply isotonic regression to CIFAR-10 VGG-19 | VGG-19 · CIFAR-10 | T-Cal verdict: 2.13% -> 1.13% | Improved |
Closest papers by meaning
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.91)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.91)
- Self-Calibrating Conformal Prediction (NeurIPS 2024 · similarity 0.89)
- The Minimax Rate of Second-Order Calibration (NeurIPS 2026 · similarity 0.88)
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.88)
- Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
