Research map / Label-efficient and robust learning / Recognition and long-tail
Overinterpretation reveals image classification model pathologies
Brandon Carter, Siddhartha Jain, Jonas Mueller, David Gifford
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train and evaluate on 5% backward-selection pixel subsets rather than full images | ResNet18 · CIFAR-10 test | accuracy (%): 95.17 ± 0.21 -> 94.96 ± 0.04 | No clear effect |
| Replace backward-selection pixel subsets with random pixel subsets (ablation) | ResNet18 trained on 5% subsets · CIFAR-10 test | accuracy (%): 94.96 ± 0.04 -> 51.27 ± 0.82 | Hurt |
| Apply input dropout to training and test images | ResNet18 · CIFAR-10-C | accuracy (%): 75.08 ± 0.20 -> 80.35 ± 0.39 | Improved |
| Apply input dropout to training and test images | ResNet18 · CIFAR-10 test | accuracy (%): 95.17 ± 0.21 -> 94.15 ± 0.26 | Hurt |
| Ensemble five independently trained ResNet18 models | ResNet18 · CIFAR-10 test | accuracy (%): 95.17 ± 0.21 -> 96.07 | Improved |
| Train and evaluate on 10% backward-selection pixel subsets rather than full images | Inception v3 · ImageNet validation | Top 1 accuracy (%): 77.21 -> 71.37 ± 0.15 | Hurt |
| Replace 10% backward-selection subsets with 10% random pixel subsets (ablation) | Inception v3 trained on subsets · ImageNet validation | Top 1 accuracy (%): 71.37 ± 0.15 -> 64.53 ± 0.16 | Hurt |
| Replace 10% backward-selection subsets with 10% random pixel subsets (ablation) | pre-trained Inception v3 · ImageNet validation | Top 1 accuracy (%): 73.87 -> 0.75 ± 0.02 | Hurt |
| Train and evaluate on 10% backward-selection subsets rather than full images | ResNet50 · ImageNet validation | Top 1 accuracy (%): 76.13 -> 65.71 ± 0.08 | Hurt |
| Replace 10% backward-selection subsets with 10% random pixel subsets (ablation) | ResNet50 trained on subsets · ImageNet validation | Top 1 accuracy (%): 65.71 ± 0.08 -> 55.70 ± 0.24 | Hurt |
| Train and evaluate on 5% backward-selection subsets without data augmentation (ablation) | CIFAR-10 classifiers · CIFAR-10 test, 5% BS subsets | accuracy (%): The paper says training without data augmentation slightly improves accuracy. | Improved |
| Calibrate with temperature scaling (ablation) | CIFAR-10 classifiers · CIFAR-10 test, SIS size | SIS size (% of image): 2.36 ± 1.21 -> 2.94 ± 1.39; 2.53 ± 1.53 -> 3.54 ± 1.94; 2.18 ± 1.37 -> 8.26 ± 2.86 | Mixed |
Closest papers by meaning
- Detecting Overfitting via Adversarial Examples (NeurIPS 2019 · similarity 0.88)
- Shallow-Deep Networks: Understanding and Mitigating Network Overthinking (ICML 2019 · similarity 0.87)
- No One Representation to Rule Them All: Overlapping Features of Training Methods (ICLR 2022 · similarity 0.86)
- When does dough become a bagel? Analyzing the remaining mistakes on ImageNet (NeurIPS 2022 · similarity 0.86)
- An Empirical Evaluation of Current Convolutional Architectures' Ability to Manage Nuisance Location and Scale Variability (CVPR 2016 · similarity 0.86)
- More is Better: when Infinite Overparameterization is Optimal and Overfitting is Obligatory (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
