Research map / Label-efficient and robust learning / OOD and anomaly detection
MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MaxEnt Loss, a Focal-loss-derived objective that adds global and local mean/variance constraints (Lagrange multipliers solved by Newton-Raphson) to improve calibration under OOD shift on CIFAR-C, TinyImageNet-C and Wilds benchmarks.
Problem
Neural network calibration degrades under out-of-distribution (OOD) shift, and existing calibration objective functions do not consistently remain well calibrated OOD.
Approach
Relates Focal loss to the Principle of Maximum Entropy and adds training-derived constraints (mean and/or variance of the label distribution, global and local) as Lagrange-multiplier terms, with multipliers estimated numerically via Newton-Raphson rather than hand-tuned.
Key results
- On CIFAR10-C, MaxEnt M reduces ECE to 8.70% vs 14.5% for CE and 9.30% for Focal, with accuracy 77.1% vs 77.9% for CE.
- On CIFAR100-C, MaxEnt M achieves ECE 6.30% vs 10.2% for CE and 8.80% for Focal.
- On Camelyon17-Wilds, MaxEnt M+V achieves ECE 8.30% vs 15.5% for CE, with accuracy 83.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add mean constraint to Focal loss | Focal loss · CIFAR10-C | ECE: 9.30$\pm{0.1}$ -> 8.70$\pm{0.2}$ | Helped |
| use variance constraint instead of mean constraint (ablation) | MaxEnt loss · CIFAR10-C | ECE: 8.70$\pm{0.2}$ -> 8.90$\pm{0.2}$ | No clear effect |
| combine mean and variance constraints (ablation) | MaxEnt loss · CIFAR10-C | ECE: 8.70$\pm{0.2}$ -> 11.5$\pm{0.5}$ | Hurt |
| add mean constraint to Focal loss | Focal loss · CIFAR100-C | ECE: 8.80$\pm{0.7}$ -> 6.30$\pm{0.2}$ | Helped |
| use variance constraint instead of mean constraint (ablation) | MaxEnt loss · CIFAR100-C | ECE: 6.30$\pm{0.2}$ -> 8.60$\pm{1.2}$ | No clear effect |
| combine mean and variance constraints (ablation) | MaxEnt loss · CIFAR100-C | ECE: 6.30$\pm{0.2}$ -> 7.30$\pm{0.5}$ | Hurt |
| add mean constraint to Focal loss | Focal loss · TinyImageNet-C | ECE: 13.9$\pm{0.1}$ -> 10.2$\pm{0.1}$ | Helped |
| use variance constraint instead of mean constraint (ablation) | MaxEnt loss · TinyImageNet-C | ECE: 10.2$\pm{0.1}$ -> 9.40$\pm{0.1}$ | Helped |
| combine mean and variance constraints (ablation) | MaxEnt loss · Camelyon17-Wilds | ECE: 12.3$\pm{0.7}$ -> 8.30$\pm{2.0}$ | Helped |
| use variance constraint instead of mean constraint (ablation) | MaxEnt loss · iWildCam-Wilds | ECE: 20.0$\pm{1.2}$ -> 25.0$\pm{0.5}$ | Hurt |
| apply temperature scaling to MaxEnt M | MaxEnt M · Camelyon17-Wilds | ECE: 12.3$\pm{0.7}$ -> $\downarrow$7.40$\pm{1.4}$ | Helped |
| apply label smoothing to MaxEnt M | MaxEnt M · CIFAR100-C | ECE: 6.30$\pm{0.2}$ -> 15.4$\pm{2.5}$ | Hurt |
Surprising findings
- Combining mean and variance constraints (M+V) does not consistently yield better calibration than single constraints.
- Soft-AvUC, when tuned by the authors, outperformed the Focal loss baseline OOD, contrary to the original paper's reported results.
Limitations
- Assumes the training label prior is aligned with the test distribution; may fail when test shifts differ greatly from training.
- Uses a single shared Lagrange multiplier for global and local constraints, with no tuned trade-off between them.
Experimental setup
Datasets: CIFAR10 CIFAR100 TinyImageNet CIFAR10-C CIFAR100-C TinyImageNet-C Camelyon17-Wilds iWildCam-Wilds FMoW-Wilds
Benchmarks: CIFAR10-C CIFAR100-C Tiny ImageNet-C Camelyon17-Wilds iWildCam-Wilds FMoW-Wilds
Metrics: accuracy ECE CECE KSE MCE AdaECE NLL Brier score
Baselines: CE Focal loss Inverse Focal loss AvUC Soft AvUC Poly loss
Builds on: Focal loss Principle of Maximum Entropy AvUC Poly loss label smoothing
Closest papers by meaning
- MaxEnt Loss: Calibrating Graph Neural Networks under Out-of-Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.88)
- Testing Calibration in Nearly-Linear Time (NeurIPS 2024 · similarity 0.85)
- Set Learning for Accurate and Calibrated Models (ICLR 2024 · similarity 0.84)
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.84)
- Optimal Multiclass U-Calibration Error and Beyond (NeurIPS 2024 · similarity 0.84)
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.