Research map / Label-efficient and robust learning / Recognition and long-tail
Large Learning Rates Simultaneously Achieve Robustness to Spurious Correlations and Compressibility
Melih Barsbey, Lucas Prieto, Stefanos Zafeiriou, Tolga Birdal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase learning rate across the sweep range (ablation) | ResNet18 · Colored MNIST | unbiased test accuracy (%): 68.343 -> 91.543 | Improved |
| increase learning rate across the sweep range (ablation) | ResNet50 · Double MNIST | unbiased test accuracy (%): 42.045 -> 95.733 | Improved |
| increase learning rate across the sweep range (ablation) | Wide ResNet101-2 · Double MNIST | unbiased test accuracy (%): 43.12 -> 96.08 | Improved |
| increase learning rate across the sweep range (ablation) | ResNet18 · MNIST-CIFAR | unbiased test accuracy (%): 26.233 -> 47.513 | Improved |
| increase learning rate across the sweep range (ablation) | Swin Transformer · CelebA (worst-group) | worst-group accuracy (%): 40.889 -> 48.21 | Improved |
| increase learning rate across the sweep range (ablation) | FCN · Colored MNIST | unbiased test accuracy (%): 72.727 -> 76.11 | Improved |
| increase learning rate with PSGD optimizer (ablation) | ResNet50 (PSGD) · Colored MNIST | unbiased test accuracy (%): 24.16 -> 93.09 | Improved |
| add L2 regularization on top of high learning rate (ablation) | ResNet18 · Colored MNIST | max unbiased test accuracy (%): 92.203 -> 94.25 | Improved |
| add L2 regularization on top of high learning rate (ablation) | ResNet18 · Double MNIST | max unbiased test accuracy (%): 96.39 -> 97.75 | Improved |
| add L2 regularization on top of high learning rate (ablation) | ResNet18 · MNIST-CIFAR | max unbiased test accuracy (%): 47.497 -> 48.327 | Improved |
| use ASAM rho sweep instead of learning rate sweep (ablation) | ResNet18 · MNIST-CIFAR | max unbiased test accuracy (%): 47.497 -> 34.053 | Hurt |
| use focal loss gamma sweep instead of learning rate sweep (ablation) | ResNet18 · MNIST-CIFAR | max unbiased test accuracy (%): 47.497 -> 40.393 | Hurt |
Closest papers by meaning
- Complexity Matters: Feature Learning in the Presence of Spurious Correlations (ICML 2024 · similarity 0.88)
- On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling (NeurIPS 2025 · similarity 0.86)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.86)
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.85)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.85)
- From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
