Research map / Label-efficient and robust learning / Recognition and long-tail
Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BalCAL, which pairs a learnable classifier with a fixed Simplex ETF classifier behind an adapter, and uses a dynamic binary-searched mixing weight to correct both overconfidence and underconfidence in model calibration.
Problem
Deep neural networks are miscalibrated, mostly overconfident but sometimes underconfident (e.g., after Mixup), and existing classifier-side calibration methods do not handle both directions well.
Approach
A fixed Simplex ETF classifier whose scaling factor controls output confidence is added via an adapter alongside the learnable classifier; the mixing weight between the two classifiers is dynamically set each epoch by matching training-set confidence to a scaled accuracy.
Key results
- On CIFAR-10, BalCAL reaches 92.23% accuracy and 0.76% ECE versus 2.18% for TST; on CIFAR-100 it reaches 4.21% ECE versus 7.07% for TST.
- Average ECE across CIFAR-10, CIFAR-100 and SVHN is 1.77, the lowest among compared methods (TST 3.23, VTST 3.11).
- On OOD detection with CIFAR-10 as in-distribution, BalCAL reports AUROC 89.89 and FPR95 64.15, better than vanilla (88.56, 70.98).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine a learnable classifier with a confidence-tunable fixed ETF classifier and dynamically adjust γ | WRN-28-10 · CIFAR-10 | ECE: 2.18±0.353 -> 0.76±0.26 | Helped |
| combine a learnable classifier with a confidence-tunable fixed ETF classifier and dynamically adjust γ | WRN-28-10 · CIFAR-100 | ECE: 7.07±0.084 -> 4.21±0.14 | Helped |
| add the adapter to the fixed ETF classifier module (ablation) | BalCAL · CIFAR-10 | ECE: 1.72(1.03) -> 0.76(0.69) | Helped |
| add the adapter to the fixed ETF classifier module (ablation) | BalCAL · CIFAR-100 | ECE: 9.34(8.79) -> 4.21(3.97) | Helped |
| add the adapter to the fixed ETF classifier module (ablation) | BalCAL · SVHN | ECE: 0.90(0.83) -> 0.24(0.59) | Helped |
| dynamically adjust γ instead of using fixed γ (ablation) | learnable classifier plus ETF classifier · CIFAR-10 | ECE | Helped |
| replace the learnable classifier with a fixed ETF classifier (β=1) (ablation) | WRN-28-10 · CIFAR-10 | ECE: 6.38(1.12) -> 66.91(0.77) | Hurt |
| replace the learnable classifier with a fixed ETF classifier (β=1) (ablation) | WRN-28-10 · CIFAR-100 | ECE: 21.63(3.10) -> 68.96(6.31) | Hurt |
| replace the learnable classifier with a fixed ETF classifier (β=1) (ablation) | WRN-28-10 · SVHN | ECE: 3.32(0.72) -> 70.76(1.27) | Hurt |
| vary Mixup interpolation parameter α | Mixup · CIFAR-10 | ECE | Hurt |
Surprising findings
- A fixed ETF classifier alone, with only the encoder trained, performs no better than the vanilla model for calibration; the dynamic mechanism applied to beta alone gives only second-best ECE.
Limitations
- Mixing weight gamma and hyperparameter delta require tuning (delta recommended in [0.91, 0.99]); beta must be set manually (1 for overconfident models, K for Mixup).
- Calibration gains on CIFAR-100-C and high-shift settings are reported only as ECE curves, with limited analysis of failure cases.
- Evaluation is limited to small-scale image benchmarks with WRN and ResNet backbones in the main text.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet CIFAR-10-C CIFAR-100-C
Metrics: accuracy ECE AECE AUROC FPR95
Baselines: Vanilla ACLS Label smoothing Mixup MIT Focal loss FLSD CPC PLP MMCE TST VTST
Builds on: Neural collapse Temperature Scaling Mixup Two-stage training (TST) Progressive Layer-Peeled training (PLP)
Closest papers by meaning
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.87)
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.85)
- Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · similarity 0.85)
- Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration (ICML 2024 · similarity 0.85)
- Calibration Bottleneck: Over-compressed Representations are Less Calibratable (ICML 2024 · similarity 0.85)
- Decoupling Feature Extraction and Classification Layers for Calibrated Neural Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.