Research map / Label-efficient and robust learning / Recognition and long-tail
Parametric ρ-Norm Scaling Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ρ-Norm Scaling, a parametric post-hoc calibrator with a learnable ρ-norm normalization of logits, trained with a multi-level loss combining bin-level squared calibration error and an instance-level KL regularizer to keep calibrated distributions close to the original softmax outputs.
Problem
Modern deep classifiers are overconfident, partly because output magnitudes grow during supervised learning; bin-level calibration objectives such as ECE are prone to overfitting and lose instance-level information when used to fit post-hoc calibrators on limited validation data.
Approach
Replace the temperature in temperature scaling with a logit normalization by a learnable ρ-norm (plus γ, β), which bounds confidence and keeps predictions order-preserving, and optimize it with a squared calibration error (SCE) term plus a KL divergence between pre- and post-calibration distributions.
Key results
- On CIFAR-100 with ResNet18, ECE drops to 0.009 for ρ-Norm Scaling versus 0.033 for temperature scaling and 0.160 uncalibrated.
- On 102 Flower with ResNet50, ECE is 0.046 versus 0.086 for temperature scaling, but MCE is not uniformly improved (e.g., SVHN ResNet18 MCE 0.438 vs 0.251 for histogram binning).
- Ablation on CIFAR-100 shows ρ-Norm Scaling beats temperature scaling under the same NLL objective (ResNet50 ECE 0.006 vs 0.048).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Temperature Scaling with ρ-Norm Scaling (ablation) | post-hoc calibration optimized with NLL · CIFAR-100, ResNet35 and ResNet50 | ECE: 0.026 -> 0.011; 0.048 -> 0.006 | Helped |
| replace Temperature Scaling with ρ-Norm Scaling (ablation) | post-hoc calibration optimized with the proposed objective · CIFAR-100, ResNet35 and ResNet50 | ECE: 0.024 -> 0.009; 0.042 -> 0.007 | Helped |
| add KL regularization to SCE (ablation) | Vector Scaling · CIFAR-100 | ECE: 0.173 -> 0.041 | Helped |
| add KL regularization to NLL (ablation) | Vector Scaling · CIFAR-100 | ECE: 0.056 -> 0.039 | Helped |
| add KL regularization to SB-ECE (ablation) | Vector Scaling · CIFAR-100 | ECE: 0.156 -> 0.039 | Helped |
| optimize with KL alone instead of a combined objective (ablation) | Vector Scaling · CIFAR-100 | ECE: 0.041 -> 0.161 | Hurt |
| set ρ to 1.75 rather than 1.5 (ablation) | ρ-Norm Scaling · CIFAR-100, ResNet35 | ECE: 0.028 -> 0.006 | Helped |
| set ρ to 1.75 rather than 1.5 (ablation) | ρ-Norm Scaling · Tiny-ImageNet, ResNet35 | ECE: 0.052 -> 0.007 | Helped |
| increase α from 1 to 10 (ablation) | ρ-Norm Scaling multi-level objective · CIFAR-100, ResNet35 | ECE: 0.041 -> 0.101 | Hurt |
| increase α from 1 to 100 (ablation) | ρ-Norm Scaling multi-level objective · 102 Flower, ResNet50 | ECE: 0.048 -> 0.102 | Hurt |
| use ρ-Norm Scaling rather than Vector Scaling | post-hoc calibration · SVHN, ResNet18 | ECE: 0.007 -> 0.008 | Hurt |
| use ρ-Norm Scaling rather than Vector Scaling | post-hoc calibration · CIFAR-10, ResNet35 | MCE: 0.077 -> 0.107 | Hurt |
Surprising findings
- MCE is not improved by the proposed method on several datasets, and on SVHN ResNet18 its MCE is worse than most baselines.
- Vector Scaling and some baselines sometimes beat the proposed method on MCE and on SVHN ECE.
Limitations
- Grid search over ρ adds bi-level optimization complexity compared with temperature scaling.
- Sensitive to the KL weight α; too large values degrade calibration.
- Limited gain for few-parameter calibrators; MCE not improved.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN 102 Flower Tiny-ImageNet
Benchmarks: CIFAR-100 CIFAR-10 SVHN 102 Flower Tiny-ImageNet
Metrics: ECE MCE AdaECE KL divergence
Baselines: Histogram Binning Temperature Scaling Vector Scaling TS-AvUC NLL-AvUC SB-ECE
Builds on: Temperature Scaling Vector Scaling Square Calibration Error AvUC loss Beta Calibration
Closest papers by meaning
- Combining Priors with Experience: Confidence Calibration Based on Binomial Process Modeling (AAAI 2025 · similarity 0.87)
- Improving Multi-Class Calibration through Normalization-Aware Isotonic Techniques (ICML 2025 · similarity 0.86)
- Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · similarity 0.86)
- Truthful Calibration Errors for Multi-Class Prediction (NeurIPS 2026 · similarity 0.86)
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.86)
- Provable Uncertainty Decomposition via Higher-Order Calibration (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.