AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set

Gabriel Laberge, Yann Pequignot, Alexandre Mathieu, Foutse Khomh, Mario Marchand

JMLR 2023 · Label-efficient and robust learning › Recognition and long-tail · approach: Feature attribution methods

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
explain the average model instead of individual models, using aggregated attributionSHAP on MLP ensemble · Toy regressionfeature ordering x1 vs x2: x2 4.59 vs x1 3.98Hurt
average models into h_EMLP ensemble · Toy regressiontest RMSE: 0.47 -> 0.43Improved
use ordinal consensus as uncertainty scoremean-rank aggregation · Toy regressionuncertainty calibration: consensus 0.83 for x1 and x2 despite a second-best model disagreeing (RMSE 0.512)Hurt
use variance as uncertainty scoreattribution aggregation · Toy regressionuncertainty calibration: variance 0.50 and 0.35 for x2 and x1 though both exceed x3 and x4 for all modelsHurt
abstain on feature pairs where good models disagree (partial order of consensus)Rashomon Set explanations · Toy regressiontrustworthy statementsImproved
group correlated features (Garage, Bsmt, AbvGrd) and use joint attribution (ablation)Additive Regression Rashomon Set · Kaggle-Housespartial order cardinalityImproved
add grouping of correlated features to local attributions (ablation)Additive Regression Rashomon Set · Kaggle-Housessign consistency of Garage and Bsmt attributionsImproved
vary tolerance epsilon around 0.1444 (ablation)Additive Regression Rashomon Set · Kaggle-Housespartial order cardinalityNo clear effect
compare consensus ranking with min-max intervals of global importance (Model Class Reliance)global feature importance · Kaggle-Housesordering of 1stFlrSF vs OverallCondImproved
increase quadrature steps by factor 10 (ablation)Integrated Gradient trapezoid · COMPASGap Error: error reduced by a factor 100Improved
include slightly worse models in the Rashomon Set (epsilon 4.23) (ablation)Kernel Ridge IG attributions · COMPASattribution of Race=BlackMixed
switch polynomial kernel to Gaussian kernel (ablation)Kernel Ridge Regression · COMPAStest RMSE: 2.12 -> 2.11No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy