Research map / Optimization / Implicit bias and generalization
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decrease batch size b in persistent SGD (b=1 to 0.3, 0.1) (ablation) | persistent SGD, single-layer net on two-cluster Gaussian mixture · two-cluster model, alpha=2, Delta=0.5 | generalization error | Improved |
| add ridge regularization (lambda 0 to 0.1, 1) to full-batch GD (ablation) | full-batch GD, two-cluster model · two-cluster model | generalization error | Improved |
| smaller mini-batch or larger ridge regularization (ablation) | SGF on three-cluster door-activation model · three-cluster model | generalization error / weight norm | Improved |
| decrease persistence time tau of the minibatch in persistent SGD (ablation) | persistent SGD · two-cluster model, b=0.3 | early-stopping generalization error | Improved |
| increase initial weight norm R (ablation) | full-batch GD · two-cluster model | early-stopping generalization error | Hurt |
| vary initialization R at long times (ablation) | full-batch GD without regularization · two-cluster model | long-time generalization error | No clear effect |
Closest papers by meaning
- Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks (NeurIPS 2022 · similarity 0.90)
- Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues (ICML 2026 · similarity 0.89)
- Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent (JMLR 2024 · similarity 0.89)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.89)
- Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks (NeurIPS 2025 · similarity 0.88)
- High-dimensional Limit of SGD for Diagonal Linear Networks (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
