AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification

Takashi Ishida, Ikko Yamane, Nontawat Charoenphakdee, Gang Niu, Masashi Sugiyama

ICLR 2023 · Label-efficient and robust learning › Recognition and long-tail · approach: Confidence calibration objectives · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Estimate Bayes error directly by averaging uncertainty labels instead of training a modelBinary classification with clean soft or uncertainty labels · Synthetic binary classificationBayes error estimation accuracyImproved
Train an MLP instead of directly averaging uncertainty labelsBinary classification · Synthetic binary classificationBayes error estimation accuracyHurt
Use the original minimum-based estimator with noisy soft labels rather than clean labels (ablation)Proposed Bayes error estimator · Synthetic binary classification with noisy labelsBayes error estimation accuracyHurt
Add sign labels and use the noise-corrected estimator (ablation)Proposed Bayes error estimator · Synthetic binary classification with noisy labelsBayes error estimation accuracyImproved
Use positive-confidence data instead of positive-negative data (ablation)Proposed Bayes error estimator · Synthetic binary classificationBayes error estimation accuracyMixed
Use the classifier's probabilistic output in the Bayes-error equation instead of directly averaging soft labelsClassifier-BER · Synthetic binary classificationBayes error estimation accuracyHurt
Use hard-label classifier probabilities rather than soft-label classifier probabilities in classifier-BER (ablation)Classifier-BER · Synthetic binary classificationBayes error estimation accuracyHurt
Compare ResNet-18 test error with the estimated Bayes errorResNet-18 · Fashion-MNIST-Herror: 3.852% (± 0.041%) vs 3.478% (± 0.079%)Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy