Research map / Optimization / Nonconvex and smooth optimization
Better generalization with less data using robust gradient descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace empirical-mean gradient (ERM-GD) with robust M-estimate gradient | gradient descent · noisy quadratic minimization, log-Normal noise, n=500, d=2 | excess risk and variance of risk | Helped |
| replace empirical-mean gradient with robust M-estimate gradient | gradient descent · noisy quadratic minimization, Gaussian noise | excess risk | No clear effect |
| robust gradient estimate instead of robust loss minimization (Brownlees et al.) | robust learning · noisy quadratic, high dimension, heavy-tailed noise | excess risk and computation time | Helped |
| robust gradient estimate instead of robust loss minimization | robust learning · low dimension, Gaussian noise | excess risk | No clear effect |
| run many more iterations than needed for convergence | robust gradient descent vs ERM-GD · noisy quadratic, small d, large n | excess risk | Helped |
| use robust gradient descent initialized at OLS instead of OLS, median-of-means methods | linear regression · regression with log-logistic and log-Normal noise | excess test RMSE | Helped |
| use median-of-means based regression (HS, Minsker) instead of OLS at very small sample sizes | regression · regression, small n | excess test RMSE | Hurt |
| use robust mini-batch gradient descent instead of SGD and SVRG | multi-class logistic regression · CIFAR-10, MNIST, protein homology | test misclassification rate | Mixed |
Closest papers by meaning
- Scaling-Up Robust Gradient Descent Techniques (AAAI 2021 · similarity 0.91)
- Efficient Generalization with Distributionally Robust Learning (NeurIPS 2021 · similarity 0.90)
- Estimating Generalization Performance Along the Trajectory of Proximal SGD in Robust Regression (NeurIPS 2024 · similarity 0.89)
- Distributionally Robust Optimization with Bias and Variance Reduction (ICLR 2024 · similarity 0.87)
- The Price of Implicit Bias in Adversarially Robust Generalization (NeurIPS 2024 · similarity 0.86)
- Multiclass Loss Geometry Matters for Generalization of Gradient Descent in Separable Classification (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.