Research map / Architectures and efficiency / RNNs and normalization
Deep Learning Through the Lens of Example Difficulty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use layer-4 k-NN probe prediction instead of final network prediction for examples ambiguous without their label | ResNet18 · CIFAR10, 100 most 'ambiguous unless label given' validation examples | accuracy: 25% -> 98% | Helped |
| switch from CE to 0-Hinge loss while also using full-batch GD with small learning rate (ablation) | VGG16 · CIFAR10 horse vs deer | mean output margin: $1.6\times 10^{1}$ -> $2.0\times 10^{-4}$ | Helped |
| switch from CE to 0-Hinge loss only (keep SGD) (ablation) | VGG16 · CIFAR10 horse vs deer | mean output margin: $1.6\times 10^{1}$ -> $6\times 10^{-2}$ | Helped |
| switch to full-batch GD with small LR only (keep CE) (ablation) | VGG16 · CIFAR10 horse vs deer | mean output margin: $1.6\times 10^{1}$ -> $1.1\times 10^{1}$ | No clear effect |
| margin-minimizing intervention (0-Hinge + GD) instead of CE + SGD (ablation) | VGG16 · CIFAR10 horse vs deer | test accuracy: 87.6% -> 69.5% | Hurt |
| reduce output margin via 0-Hinge loss and GD (ablation) | VGG16 · CIFAR10 | prediction depth | Hurt |
Closest papers by meaning
- Deep Learning on a Data Diet: Finding Important Examples Early in Training (NeurIPS 2021 · similarity 0.89)
- Learning explanations that are hard to vary (ICLR 2021 · similarity 0.88)
- An Empirical Study of Example Forgetting during Deep Neural Network Learning (ICLR 2019 · similarity 0.88)
- Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth (ICLR 2021 · similarity 0.87)
- Understanding deep learning requires rethinking generalization (ICLR 2017 · similarity 0.87)
- FaCT: Faithful Concept Traces for Explaining Neural Network Decisions (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.