AI Paper Atlas

Research map / Architectures and efficiency / CNN design

Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency

Robert Geirhos, Kristof Meding, Felix A. Wichmann

NeurIPS 2020 · Architectures and efficiency › CNN design · approach: Biologically inspired CNN explanation · 23 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use CNNs instead of human observers as the compared decision makersImageNet-trained CNNs · Cue conflict, edge, and silhouette stimulierror consistency (κ)Improved
Compare ImageNet-trained CNNs with human observersCNNs and human observers · Cue conflict and edge stimulierror consistency (κ)Hurt
Increase ImageNet validation accuracyCNNs compared with humans · Cue conflicterror consistency (κ): F(1, 158) = 0.086, p=.769, R^2=0.001No clear effect
Increase ImageNet validation accuracyCNNs compared with humans · Line drawing stimulierror consistency (κ): F(1, 158) = 0.478, p=.491, R^2=0.003No clear effect
Increase ImageNet accuracyCNNs compared with humans · Silhouette stimulierror consistency (κ): F(1, 158) = 53.530, p=1.21\cdot 10^{-11}, R^2=0.253Improved
Increase top-5 ImageNet accuracyCNNs compared with humans · ImageNet imageserror consistency (κ): F(1, 30) = 8.162, p=.008, R^2=0.214Hurt
Use a recurrent CORnet-S instead of a feedforward ResNet-50CNN-human comparison · Object recognition behavioural stimulierror consistency (κ): .068: ResNet-50; .066: CORnet-SNo clear effect
Use CORnet-S rather than ResNet-50CNN-CNN comparison · Object recognition behavioural stimulierror consistency (κ): .711Improved
Increase model shape bias through stylized training dataCNNs compared with humans · Cue conflict stimulierror consistency (κ): .066, .068, .098, .195Improved
Compare CNNs with humans on silhouette stimuli rather than cue conflict or line-drawing stimuliCNN-human error consistency · Silhouette stimulierror consistency (κ)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy