AI Paper Atlas

Research map / Large language models / AI and society

HIVE: Evaluating the Human Interpretability of Visual Explanations

Sunnie S. Y. Kim, Nicole Meister, Vikram V. Ramaswamy, Ruth Fong, Olga Russakovsky

ECCV 2022 · Large language models › AI and society · approach: Human-centered trust and agency · 94 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
provide explanations to participants (agreement task)GradCAM, BagNet, ProtoPNet, ProtoTree · CUB agreementbelief that incorrect predictions are correct: 67.2%, 57.6%, 53.6%, 62.8% of incorrect predictions believed correctHurt
provide explanations on ImageNet (agreement task)GradCAM, BagNet · ImageNet agreementbelief that incorrect predictions are correct: 55.2% and 64.4%Hurt
switch from correct to incorrect predictions in distinction taskGradCAM · CUB distinctionparticipant accuracy (%): 71.2 -> 26.4Hurt
switch from correct to incorrect predictions in distinction taskBagNet · CUB distinctionparticipant accuracy (%): 45.6 -> 32.0Hurt
show ground-truth class labels (ablation)GradCAM and BagNet · ImageNet distinctionparticipant accuracy (%): 51.2 -> 49.2 (GradCAM), 38.4 -> 34.8 (BagNet)Hurt
compare model similarity scores with human similarity ratings (ProtoTree)ProtoTree · CUBPearson correlation: 0.06No clear effect
compare model similarity scores with human ratings (ProtoPNet)ProtoPNet · CUBSpearman correlation: rho=-0.25, p=0.49 (distinction); rho=-0.52, p=0.12 (agreement)No clear effect
replace similarity-rating agreement task with selecting first disagreeing tree step (ablation)ProtoTree · CUB agreementparticipant accuracy (%): 53.6 -> 52.8No clear effect
use automatic localization metrics to predict human agreement confidenceGradCAM · CUB/ImageNet agreementcorrelation with human confidenceNo clear effect
show participants their task performance before final self-ratingself-rated understanding · all studiesself-rated understanding (1-5): 3.8 -> 3.5Hurt
increase deployment risk level in tradeoff questionbaseline vs explained model · tradeoff questionsrequired extra baseline accuracy: +6.2% -> +8.2% -> +10.9%Improved
compare ProtoTree on distinction against othersProtoTree · CUB distinction (correct)participant accuracy (%): 71.2 (GradCAM) vs 33.8Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy