AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis

Thomas FEL, Remi Cadene, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, Thomas Serre

NeurIPS 2021 · Probabilistic and scientific modeling › Molecules and proteins · approach: Sparse concepts and hypothesis testing · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use total Sobol indices to include higher-order interactions instead of first-order indices alone (ablation)Sobol attribution · ImageNet validationDeletion: 0.170 -> 0.121Improved
use signed total Sobol indices instead of standard total Sobol indices (ablation)Sobol attribution · VOC07 Test and COCO14 ValPointing Game accuracy: 87.7 -> 89.8; 75.3 -> 82.4; 54.6 -> 55.2; 49.6 -> 49.9; 89.6 -> 86.7; 80.2 -> 73.0; 57.3 -> 57.5; 52.7 -> 53.1Mixed
use signed total Sobol indices instead of standard total Sobol indices (ablation)Sobol attribution · ImageNet validationDeletion: 0.121 -> 0.145; 0.109 -> 0.114; 0.104 -> 0.147; 0.107 -> 0.141Hurt
reduce the number of forwards from the default 3936 to 492 (ablation)Sobol attribution on ResNet50V2 · ImageNet validationDeletion: 0.121 -> 0.151Hurt
reduce the number of forwards from 3936 to 2460 (ablation)Sobol attribution on ResNet50V2 · ImageNet validationDeletion: 0.121 -> 0.121No clear effect
lower the word-mask threshold to 0.05 so fewer words are removed (ablation)Sobol signed on BERT · IMDB word deletionWord deletion: 0.598 -> 0.527Improved
use Sobol rather than RISEBlack-box attribution · ImageNet validation, EfficientNet convergenceSpearman rank correlation: 0.65 -> 0.8 after 1,000 forwards; 0.73 -> 0.97 with 7,000 forwardsImproved
use Sobol rather than RISEBlack-box attribution · Pointing Game benchmarksPointing Game accuracy: RISE to Sobol: VOC VGG16 86.9 to 87.7, ResNet50 86.4 to 89.6; COCO VGG16 50.8 to 54.6, ResNet50 54.7 to 57.3Improved
use Sobol rather than RISEBlack-box attribution · ImageNet validationDeletion: 0.127 -> 0.121; 0.121 -> 0.109; 0.119 -> 0.104; 0.114 -> 0.107Improved
use Sobol signed rather than standard SobolBlack-box attribution · IMDB word deletionWord deletion: 0.662 -> 0.598; 0.523 -> 0.461Improved
use Sobol signed rather than standard SobolBlack-box attribution · ImageNet validationInsertion: 0.370 -> 0.258; 0.313 -> 0.290; 0.309 -> 0.204; 0.331 -> 0.211Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy