AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Unsupervised Evaluation and Weighted Aggregation of Ranked Classification Predictions

Mehmet Eren Ahsen, Robert M Vogel, Gustavo A Stolovitzky

JMLR 2019 · Label-efficient and robust learning › Recognition and long-tail · approach: Learning from unlabeled and positive-unlabeled data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use rank transformation instead of raw confidence scores (ablation)SUMMA · BCL6 DREAM Challenge (11 methods)AUC: 0.83 -> 0.93Improved
weight classifiers by SUMMA-inferred AUC instead of plain rank averagingensemble · Synthetic (M=30, N=1000)AUC: 0.89 -> 0.95Improved
weight classifiers by SUMMA-inferred AUC instead of plain rank averagingensemble · BCL6 DREAM Challenge (11 methods)AUC: 0.82 -> 0.93Improved
increase number of base classifiers from 5 to 15+SUMMA AUC estimation · Syntheticcorrelation: 0.87 -> 0.97Improved
increase number of samples from 30 to 4000SUMMA AUC estimation · Syntheticcorrelation: 0.57 -> 1Improved
use SUMMA instead of WOC with very few samples (N<=50)ensemble · SyntheticAUCHurt
extreme class prevalence (<=0.2 or >=0.8)SUMMA · SyntheticAUC estimation accuracyHurt
SUMMA labels vs SML binarized outputsclassification · BCL6 DREAM Challenge (11 methods)balanced accuracy: 0.80 -> 0.82Improved
SUMMA vs k-means clustering of scoresclassification · BCL6 DREAM Challenge (11 methods)balanced accuracy: 0.52 -> 0.82Improved
add more classifiers (5 to 10) when comparing SUMMA with WOCSUMMA · UCI Bank Marketing% SUMMA better than WOC: 90 -> 99Improved
SUMMA vs WOC on Yeast data with more classifiersSUMMA · UCI Yeast% SUMMA better than WOC: 69 -> 68No clear effect
use SUMMA instead of best individual classifierensemble · UCI datasetsAUC rankMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy