AI Paper Atlas

Research map / Label-efficient and robust learning / OOD and anomaly detection

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles

Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, Yingyu Liang, Somesh Jha

NeurIPS 2021 · Label-efficient and robust learning › OOD and anomaly detection · approach: Uncertainty and misclassification detection · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace Proxy Risk with self-training ensemble (TRM) for accuracy estimationProxy Risk · Digitsabsolute estimation error: 0.085±0.142 -> 0.023±0.020Improved
replace Proxy Risk with TRM for error detectionProxy Risk · DigitsF1 score: 0.844±0.118 -> 0.881±0.084Improved
replace Proxy Risk with TRM for error detectionProxy Risk · Office-31F1 score: 0.605±0.177 -> 0.767±0.052Improved
use TRI ensemble instead of TRM on imbalanced iWildCamOurs · iWildCamabsolute estimation error: 0.035±0.022 -> 0.015±0.008Improved
use TRI instead of TRM on DigitsOurs · Digitsabsolute estimation error: 0.023±0.020 -> 0.164±0.218Hurt
use TRI instead of TRM on Amazon ReviewOurs · Amazon ReviewF1 score: 0.426±0.036 -> 0.384±0.032Hurt
replace Proxy Risk with TRM on Office-31 accuracy estimationProxy Risk · Office-31absolute estimation error: 0.033±0.012 -> 0.029±0.021No clear effect
replace Proxy Risk with TRM on iWildCam accuracy estimationProxy Risk · iWildCamabsolute estimation error: 0.119±0.043 -> 0.035±0.022Hurt
replace average confidence with TRM on CIFAR10-CAvg Conf · CIFAR10-Cabsolute estimation error: 0.353±0.175 -> 0.022±0.009Improved
use DANN-arch instead of typical DNN as model fOurs (RM) · DigitsF1 score: 0.881±0.084 -> 0.841±0.112Hurt
increase ensemble size N (N=1 means no ensemble) (ablation)Ours (TRM) · Digitsaverage F1 scoreImproved
increase self-training iterations T (T=1 means no self-training) (ablation)Ours (TRM) · Digitsaverage F1 scoreImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy