Research map / Optimization / Implicit bias and generalization
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stochastic mirror descent with a step size making ψ(w) − ηLᵢ(w) convex | Stochastic mirror descent · General nonlinear models and loss functions, minimax problem | optimal minimax value: 1 | Helped |
| Use SGD with step size 0 < η ≤ minᵢ 1/||xᵢ||² | SGD · Linear models with square loss, minimax problem | optimal minimax value: 1 | Helped |
| Choose the mirror potential ψ to encode a desired regularizer | Stochastic mirror descent · Over-parameterized linear models | distance to the selected regularized solution | Helped |
| Use ψ(w) = ||w||₁.₁ as the mirror potential | Stochastic mirror descent · Compressed sensing (n=50, m=100, sparsity k=10) | iterations to recover the sparse solution: around 10,000 iterations | Helped |
Closest papers by meaning
- Stochastic Composite Mirror Descent: Optimal Bounds with High Probabilities (NeurIPS 2018 · similarity 0.91)
- Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently (NeurIPS 2022 · similarity 0.91)
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (ICLR 2017 · similarity 0.90)
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (NeurIPS 2019 · similarity 0.90)
- Efficient Mirror Descent Ascent Methods for Nonsmooth Minimax Problems (NeurIPS 2021 · similarity 0.90)
- The Statistical Complexity of Early-Stopped Mirror Descent (NeurIPS 2020 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.