AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

T-Cal: An Optimal Test for the Calibration of Predictive Models

Donghwan Lee, Xinmeng Huang, Hamed Hassani, Edgar Dobriban

JMLR 2023 · Learning theory and algorithms › Prediction and decision losses · approach: Probabilistic calibration · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use debiased l2-ECE statistic instead of plug-in biased l2 (ablation)T-Cal · synthetic mis-calibration (P_1,m)type II errorImproved
use l2-ECE statistic instead of plug-in l1-ECE (ablation)T-Cal · synthetic mis-calibration (P_1,m)type II errorImproved
T-Cal versus classical logistic-refit and consistency-resampling testsT-Cal · synthetic mis-calibration (P_1,m), n=10,000true detection rateImproved
increase sample size n from 2,000 to 10,000T-Cal · synthetic mis-calibration (P_1,m)type II error: n = 2,000, 5,000, 10,000Improved
apply Platt scaling post-hoc to CIFAR-100 ResNet 56ResNet 56 · CIFAR-100empirical l1-ECE / T-Cal verdict: 15.2% -> 1.84%Improved
compare lower l1-ECE with test verdictResNet 56 · CIFAR-100T-Cal verdict: 1.84% accepted vs 1.57% rejectedMixed
apply polynomial scaling to ImageNet DenseNet 161DenseNet 161 · ImageNetempirical l1-ECE / T-Cal verdict: 5.67% -> 0.62%Improved
apply Platt scaling to CIFAR-10 DenseNet 121DenseNet 121 · CIFAR-10empirical l1-ECE: 2.02% -> 2.32%Hurt
apply isotonic regression to CIFAR-10 VGG-19VGG-19 · CIFAR-10T-Cal verdict: 2.13% -> 1.13%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy