AI Paper Atlas

Research map / Large language models / AI and society

What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods

Julien Colin, Thomas FEL, Remi Cadene, Thomas Serre

NeurIPS 2022 · Large language models › AI and society · approach: Human-centered trust and agency

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add Grad-CAM explanations during trainingHuman meta-predictors · Husky vs. WolfUtility: 0.95 -> 1.34Improved
Add SmoothGrad explanations during trainingHuman meta-predictors · Husky vs. WolfUtility: 1.20Improved
Add Occlusion explanations during trainingHuman meta-predictors · Husky vs. WolfUtility: 1.22Improved
Add Saliency explanations during trainingHuman meta-predictors · Husky vs. WolfUtility: 1.06No clear effect
Add Saliency explanations during trainingHuman meta-predictors · LeavesUtility: 1.13Improved
Add SmoothGrad explanations during trainingHuman meta-predictors · LeavesUtility: 1.13Improved
Add Grad-CAM or Occlusion explanations during trainingHuman meta-predictors · LeavesUtility: 1.10No clear effect
Add any attribution method during trainingHuman meta-predictors · ImageNet (Kit Fox vs Red Fox)Utility: 0.90 - 1.00No clear effect
Replace real explanation with non-informative bottom-up saliency (control) (ablation)Human meta-predictors · Husky vs. WolfUtility: 0.95Hurt
Replace real explanation with non-informative control (ablation)Human meta-predictors · ImageNet (Kit Fox vs Red Fox)session 3 accuracy: 58.8 -> 48.5Hurt
Use more faithful attribution methods (higher 1 - Deletion)Attribution methods · All three datasetsUtilityNo clear effect
Use Insertion and muFidelity instead of Deletion as faithfulness metricAttribution methods · All three datasetscorrelation with UtilityNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy