Research map / Label-efficient and robust learning / Recognition and long-tail
Beyond Confidence: Reliable Models Should Also Consider Atypicality
Mert Yuksekgonul, Linjun Zhang, James Zou, Carlos Guestrin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add input atypicality-dependent recalibration and class-specific corrections | ImageNet ResNet152 · ImageNet, atypicality quantile 1.0 (GMM) | ECE: 0.033 ± 0.004 -> 0.026 ± 0.001 | Improved |
| Add input atypicality-dependent recalibration and class-specific corrections | ImageNet ResNet18 · ImageNet, atypicality quantile 1.0 (GMM) | ECE: 0.032 ± 0.003 -> 0.017 ± 0.002 | Improved |
| Add input atypicality-dependent recalibration and class-specific corrections | CIFAR10 WideResNet28 · CIFAR10, atypicality quantile 1.0 (GMM) | ECE: 0.046 ± 0.002 -> 0.041 ± 0.002 | Improved |
| Add input atypicality-dependent recalibration and class-specific corrections | CIFAR100 WideResNet28 · CIFAR100, atypicality quantile 1.0 (GMM) | ECE: 0.101 ± 0.004 -> 0.109 ± 0.005 | Hurt |
| Use atypicality-aware rather than ordinary APS thresholds | APS on ResNet50 · ImageNet, fitted to 95% coverage | average prediction set size: 21.3 -> 15.6 | Improved |
| Use atypicality-aware rather than ordinary RAPS thresholds | RAPS on ResNet50 · ImageNet, fitted to 95% coverage | average prediction set size: 4.2 -> 9.1 | Hurt |
| Use atypicality-aware prediction-set thresholds | APS and RAPS · ImageNet high-atypicality and low-confidence groups | conditional coverage | Improved |
| Recalibrate with atypicality without using group annotations | Skin lesion classifier · Fitzpatrick17k skin-tone groups | worst-group accuracy: 0.69 vs 0.63 | Improved |
| Condition temperature scaling on atypicality groups instead of using one temperature | Temperature Scaling · Atypicality groups | calibration | Mixed |
| Use atypicality to stratify prediction-quality analysis | Discriminative classifiers and Alpaca-7B · Balanced, imbalanced, and LLM classification | accuracy | Hurt |
| Use atypicality to stratify prediction-quality analysis | Discriminative classifiers and Alpaca-7B · Balanced, imbalanced, and LLM classification | overconfidence | Hurt |
Closest papers by meaning
- Improving Confidence Estimates for Unfamiliar Examples (CVPR 2020 · similarity 0.86)
- Leveraging unlabeled data to predict out-of-distribution performance (ICLR 2022 · similarity 0.86)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.85)
- Rethinking Robustness of Model Attributions (AAAI 2024 · similarity 0.85)
- Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs (ICML 2024 · similarity 0.84)
- Leveraging Uncertainty Estimates To Improve Classifier Performance (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
