Research map / Label-efficient and robust learning / Recognition and long-tail
From Classification Accuracy to Proper Scoring Rules: Elicitability of Probabilistic Top List Predictions
Johannes Resin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use true top-1 list instead of mode prediction | padded symmetric scores · 5-class example | expected padded Brier score, p(m): 1 -> 0.6875 (70.59% -> 17.28% above optimal) | Improved |
| use true top-2 list instead of top-1 list | padded symmetric scores · 5-class example | expected padded Brier score, p(m): 0.6875 -> 0.5867 | Improved |
| use true top-2 list instead of top-1 list | padded symmetric scores · 5-class example | expected padded logarithmic score, p(m): 1.3863 -> 1.0532 | Improved |
| use true top-2 list instead of top-1 list | padded symmetric scores · 5-class example | expected padded Brier score, p(l): 0.7969 -> 0.7955 | No clear effect |
| use padded logarithmic instead of padded Brier score for top-1 list | padded symmetric scores · 5-class example | relative deviation from optimal score, p(h): 0.38% -> 24.75% | Hurt |
| use probabilistic top-1 list instead of hard mode | padded symmetric scores · 5-class example | expected padded logarithmic score: infinity -> 0.0699 | Improved |
Closest papers by meaning
- Leveraging Uncertainty Estimates To Improve Classifier Performance (ICLR 2024 · similarity 0.82)
- Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions (NeurIPS 2025 · similarity 0.81)
- Second-Order Uncertainty Quantification: A Distance-Based Approach (ICML 2024 · similarity 0.81)
- Foundations of the Theory of Performance-Based Ranking (CVPR 2025 · similarity 0.80)
- To Each Metric Its Decoding: Post-Hoc Optimal Decision Rules of Probabilistic Hierarchical Classifiers (ICML 2025 · similarity 0.80)
- Statistical Multicriteria Benchmarking via the GSD-Front (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
