Research map / Label-efficient and robust learning / Recognition and long-tail
Unsupervised Evaluation and Weighted Aggregation of Ranked Classification Predictions
Mehmet Eren Ahsen, Robert M Vogel, Gustavo A Stolovitzky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use rank transformation instead of raw confidence scores (ablation) | SUMMA · BCL6 DREAM Challenge (11 methods) | AUC: 0.83 -> 0.93 | Improved |
| weight classifiers by SUMMA-inferred AUC instead of plain rank averaging | ensemble · Synthetic (M=30, N=1000) | AUC: 0.89 -> 0.95 | Improved |
| weight classifiers by SUMMA-inferred AUC instead of plain rank averaging | ensemble · BCL6 DREAM Challenge (11 methods) | AUC: 0.82 -> 0.93 | Improved |
| increase number of base classifiers from 5 to 15+ | SUMMA AUC estimation · Synthetic | correlation: 0.87 -> 0.97 | Improved |
| increase number of samples from 30 to 4000 | SUMMA AUC estimation · Synthetic | correlation: 0.57 -> 1 | Improved |
| use SUMMA instead of WOC with very few samples (N<=50) | ensemble · Synthetic | AUC | Hurt |
| extreme class prevalence (<=0.2 or >=0.8) | SUMMA · Synthetic | AUC estimation accuracy | Hurt |
| SUMMA labels vs SML binarized outputs | classification · BCL6 DREAM Challenge (11 methods) | balanced accuracy: 0.80 -> 0.82 | Improved |
| SUMMA vs k-means clustering of scores | classification · BCL6 DREAM Challenge (11 methods) | balanced accuracy: 0.52 -> 0.82 | Improved |
| add more classifiers (5 to 10) when comparing SUMMA with WOC | SUMMA · UCI Bank Marketing | % SUMMA better than WOC: 90 -> 99 | Improved |
| SUMMA vs WOC on Yeast data with more classifiers | SUMMA · UCI Yeast | % SUMMA better than WOC: 69 -> 68 | No clear effect |
| use SUMMA instead of best individual classifier | ensemble · UCI datasets | AUC rank | Mixed |
Closest papers by meaning
- Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation (NeurIPS 2025 · similarity 0.85)
- Generalization Performance of Ensemble Clustering: From Theory to Algorithm (ICML 2025 · similarity 0.85)
- MEPSI: An MDL-Based Ensemble Pruning Approach with Structural Information (AAAI 2024 · similarity 0.84)
- Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective (AAAI 2024 · similarity 0.84)
- Evaluating multiple models using labeled and unlabeled data (NeurIPS 2025 · similarity 0.84)
- Exact power indices for plurality-voting ensembles (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
