Research map / Optimization / Implicit bias and generalization
On the Double Descent of Random Features Models Trained with SGD
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train random features regression with averaged SGD rather than the minimum-norm solution | Random features regression with Gaussian kernel · MNIST (digit 3 vs. 7) | test MSE | Mixed |
| Increase the number of random features across the interpolation threshold | Random features regression trained with averaged SGD · MNIST (digit 3 vs. 7) | test MSE | Mixed |
| Increase the number of random features | Random features regression trained with SGD · Under- and over-parameterized regimes | bias: n^{\zeta-1} | Helped |
| Increase the number of random features through and beyond the interpolation threshold | Random features regression trained with SGD · Under- and over-parameterized regimes | variance: mn^{\zeta-1} ,~\mbox{if $m \leqslant n$}; 1+n^{\zeta-1} + \frac{n}{m} ,~\mbox{if $m > n$} | Mixed |
| Use constant step size instead of polynomial-decay step size | Random features regression trained with averaged SGD · Under- and over-parameterized regimes | excess-risk convergence rate: \mathcal{O}(1/n) | Helped |
| Use polynomial-decay step size instead of constant step size | Random features regression trained with averaged SGD · Under- and over-parameterized regimes | learning rate | Hurt |
Closest papers by meaning
- Learning Curves for SGD on Structured Features (ICLR 2022 · similarity 0.89)
- The Benefits of Implicit Regularization from SGD in Least Squares Problems (NeurIPS 2021 · similarity 0.89)
- Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime (NeurIPS 2022 · similarity 0.89)
- Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization (ICLR 2019 · similarity 0.89)
- Estimating Generalization Performance Along the Trajectory of Proximal SGD in Robust Regression (NeurIPS 2024 · similarity 0.88)
- Statistical Guarantees for High-Dimensional Stochastic Gradient Descent (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.