Research map / Learning theory and algorithms / Prediction and decision losses
Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes angular calibration, which interpolates a linear classifier's logits with Gaussian noise (chance predictor) using the estimated angle to the true weight, proving calibration and Bregman optimality in proportional high dimensions, and shows Platt scaling converges to it.
Problem
Calibrating linear binary classifiers sigma(w_hat^T x) under Gaussian design in the high-dimensional proportional regime (n and d diverge at comparable rates), where logistic regression is known to be miscalibrated.
Approach
Interpolate the estimator's logit with independent Gaussian noise, with weights set by the angle between w_hat and the true weight w_star; this yields a predictor that is exactly calibrated for the true angle, Bregman-optimal among functions of the logit, and whose angle can be consistently estimated via AMP-style observable estimators plus a holdout sign estimator. Also shows Platt scaling conver
Key results
- On pretrained-feature tasks, angular calibration reaches ECE 0.0199 on ResNet-34/CIFAR-10 vs 0.1236 uncalibrated and 0.0259 for Platt with 500 holdout.
- On UCI Communities & Crime, angular ECE 0.0296 vs 0.1700 uncalibrated and 0.0262 for Platt 500.
- In simulation, the estimated inner product is 0.4356 vs true 0.4526, and sign estimation error was 0.89% with n_ho=100 over 5000 trials.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace uncalibrated predictions with Angular calibration | ResNet-34 · CIFAR-10 | ECE: 0.1236 -> 0.0199 | Helped |
| Replace uncalibrated predictions with Angular calibration | MiniLM · 20 Newsgroups | ECE: 0.1392 -> 0.0249 | Helped |
| Replace uncalibrated predictions with Angular calibration | ChemBERTa · Tox21 | ECE: 0.1389 -> 0.0132 | Helped |
| Replace uncalibrated predictions with Angular calibration | Linear predictor · Communities & Crime | ECE: 0.1700 -> 0.0296 | Helped |
| Replace uncalibrated predictions with Angular calibration | Linear predictor · Splice-junction Gene Seq | ECE: 0.1262 -> 0.0208 | Helped |
| Replace uncalibrated predictions with Angular calibration | Linear predictor · Madelon | ECE: 0.1473 -> 0.0267 | Helped |
| Use Angular calibration instead of Platt 500 | ResNet-34 · CIFAR-10 | ECE: 0.0259 -> 0.0199 | Helped |
| Use Angular calibration instead of Platt 500 | MiniLM · 20 Newsgroups | ECE: 0.0679 -> 0.0249 | Helped |
| Use Angular calibration instead of Platt 500 | ChemBERTa · Tox21 | ECE: 0.0175 -> 0.0132 | Helped |
| Use Angular calibration instead of Platt 500 | Linear predictor · Communities & Crime | ECE: 0.0262 -> 0.0296 | Hurt |
| Use Angular calibration instead of Platt 500 | Linear predictor · Splice-junction Gene Seq | ECE: 0.0556 -> 0.0208 | Helped |
| Use Angular calibration instead of Platt 500 | Linear predictor · Madelon | ECE: 0.0299 -> 0.0267 | Helped |
Surprising findings
- Plug-in angle estimator based on Wigner-type random matrix assumptions is unstable on real datasets, so the semi-real experiments use the true angle instead.
- Platt scaling with 100 holdout points fails to calibrate in simulation, and Isotonic regression is sometimes worse than Platt, though both improve with 500 holdout points.
Experimental setup
Datasets: CIFAR-10 20 Newsgroups Tox21 Communities & Crime Splice-junction gene sequences Madelon 20 additional UCI/OpenML datasets
Benchmarks: CIFAR-10 20 Newsgroups Tox21 Communities & Crime Splice-junction gene sequences Madelon
Metrics: expected calibration error (ECE) reliability diagrams
Baselines: Uncalibrated logistic ridge Platt scaling (100 holdout) Platt scaling (500 holdout) Isotonic regression (100 holdout) Isotonic regression (500 holdout)
Builds on: Platt scaling M-estimation high-dimensional asymptotics Bregman divergence projection theorem
Closest papers by meaning
- The Minimax Rate of Second-Order Calibration (NeurIPS 2026 · similarity 0.89)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.86)
- Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration (ICML 2024 · similarity 0.84)
- Divide et Calibra: Multiclass Local Calibration via Vector Quantization (NeurIPS 2026 · similarity 0.84)
- Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction (ICML 2025 · similarity 0.84)
- Confidence Calibration of Classifiers with Many Classes (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.