Research map / Learning theory and algorithms / Sample complexity
Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes soft-label-based estimators of the Bayes-optimal balanced error rate (BER) and AUC, handling known and unknown class priors and order-preserving corrupted soft labels, and extends the FeeBee evaluation framework to score any such estimator without knowing the true optimum.
Problem
Estimating the Bayes-optimal BER and AUC for binary classification is underexplored, and evaluating such estimators on real datasets is hard because the true optimum is unobservable.
Approach
Plug-in estimators of BER* and AUC* from soft labels; under corruption, recover clean soft labels via isotonic regression with auxiliary hard labels and estimate the prior with a clipped mean of hard labels; extend FeeBee by using a closed-form relation between optima and noise-injected optima to score estimators without the true optimum.
Limitations
- Corruption model is simple and does not cover arbitrary instance-dependent corruption
- Analysis limited to BER and AUC; non-decomposable metrics such as F-measure left to future work
- Multi-class extension not addressed
Experimental setup
Metrics: balanced error rate AUC estimation error
Builds on: FeeBee Bayes error estimation from soft labels
Closest papers by meaning
- SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning (ICML 2024 · similarity 0.87)
- Interplay of ROC and Precision-Recall AUCs: Theoretical Limits and Practical Implications in Binary Classification (ICML 2024 · similarity 0.87)
- A Closer Look at AUROC and AUPRC under Class Imbalance (NeurIPS 2024 · similarity 0.86)
- Positive-unlabeled AUC Maximization under Covariate Shift (ICML 2025 · similarity 0.86)
- Leveraging Uncertainty Estimates To Improve Classifier Performance (ICLR 2024 · similarity 0.86)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.