AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Selective Ensembles for Consistent Predictions

Emily Black, Klas Leino, Matt Fredrikson

ICLR 2022 · Label-efficient and robust learning › Recognition and long-tail · approach: Group-robust classification · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Selective ensembling (abstaining when the mode is not significant at alpha=0.05) with 5-20 constituentsMLPs/LeNet/ResNet50 trained over random seeds or leave-one-out data · seven datasets (tabular and image)percentage of test points with pflip>0 (prediction disagreement): from up to 57% (singleton, German Credit) and 5-10% typically to 0.0Improved
Ensembling without abstention (non-selective, n=5 to 20) (ablation)deep models trained over random seeds · German Creditpercentage of test points with pflip>0: from .570 (singleton) to .155 (n=20), but not eliminatedImproved
Increasing selective ensemble size from 5 to 20 models (ablation)selective ensembles (alpha=0.05) · German Creditabstention rate: abstention drops from 1.0 (n=5) to .165 (n=20) for random seedImproved
Selective abstention on unstable points within ensemblesensembles of 15 models · German Credit attributionsSpearman's rho between saliency mapsImproved
Ensembling (non-selective) of saliency-map modelsdeep models · tabular and image datasetsSpearman rank correlation of attributionsImproved
Selective ensembling with alpha=0.05 vs non-selective ensembles on accuracyselective ensembles with 20 models · Adult, Seizure, Taiwanese, FMNIST, Warfarin, Colonaccuracy (abstain as error): typically within a few points of non-selective ensemble accuracyNo clear effect
Selective ensembling on subgroups (demographic groups)selective ensembles · Adult, German Credit, Taiwanese Credit, Warfarinaccuracy disparity between groups: within 1% of original disparityNo clear effect
Lowering alpha from 0.05 to 0.01 in selective ensembles (ablation)selective ensembles with n=10-20 · tabular and image datasetsabstention rate and accuracy: e.g. German Credit n=20 abstention .165 -> .270 (RS); accuracy .664 -> .593 at n=20Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy