Research map / Label-efficient and robust learning / Recognition and long-tail
Don’t blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy
Aahlad Manas Puli, Lily H Zhang, Yoav Wald, Rajesh Ranganath
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add confidence control (SD, sigma-damp, norm, sigma-stitch) to ERM | ERM · Waterbirds, CelebA, CivilComments | test worst-group accuracy: 3-10% on Waterbirds, 4-9% on CelebA, 7-10% on CivilComments | Improved |
| Use spectral decoupling instead of ERM | ERM · Waterbirds | test worst-group accuracy: 70.8 -> 80.5 | Improved |
| Use confidence control instead of CnC | CnC · Waterbirds | test worst-group accuracy: 68.0 -> 80.5 | Improved |
| Use SD instead of CnC | CnC · CivilComments | test worst-group accuracy: 68.8 -> 70.1 | Improved |
| Use confidence control instead of JTT | JTT · CelebA | test worst-group accuracy: 75.2 -> 81.4 | Improved |
| Use large learning rate and weight decay in ERM (ablation) | ERM log-loss · CelebA | test worst-group accuracy: 56% -> 72.8% | Improved |
| Shrink validation set to 99 samples (1/200) (ablation) | ERM · CelebA | test worst-group accuracy: more than 20% | Hurt |
| Use sigma-stitch with only 99 validation samples (ablation) | sigma-stitch · CelebA | test worst-group accuracy: 76.2% | Improved |
| Use confidence control instead of ERM in pure setting (no group labels) | ERM · CelebA and Waterbirds | test worst-group accuracy: 3-8% | Improved |
| Use SD instead of ERM in pure setting | ERM · Waterbirds pure setting | test worst-group accuracy: 70.9 -> 78.7 | Improved |
| Use SD_log instead of ERM in pure setting | ERM · CivilComments pure setting | test worst-group accuracy: 60.8 -> 67.4 | Improved |
| Use SD instead of ERM in pure setting on CivilComments | ERM · CivilComments pure setting | test worst-group accuracy: 60.8 -> 60.4 | No clear effect |
Closest papers by meaning
- On the Foundations of Shortcut Learning (ICLR 2024 · similarity 0.89)
- Navigate Beyond Shortcuts: Debiased Learning Through the Lens of Neural Collapse (CVPR 2024 · similarity 0.87)
- Improved Training Technique for Shortcut Models (NeurIPS 2025 · similarity 0.85)
- Do ImageNet-trained Models Learn Shortcuts? The Impact of Frequency Shortcuts on Generalization (CVPR 2025 · similarity 0.84)
- Adaptive Shortcut Debiasing for Online Continual Learning (AAAI 2024 · similarity 0.84)
- Shortcut Features as Top Eigenfunctions of NTK: A Linear Neural Network Case and More (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
