Research map / Optimization / Implicit bias and generalization
A Unified Approach to Controlling Implicit Regularization via Mirror Descent
Haoyuan Sun, Khashayar Gatmiry, Kwangjun Ahn, Navid Azizan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use normalized step size (divide gradient by loss, decaying step) in p-GD | p-GD p=1.5, fully connected net · MNIST test | test accuracy (%): 97.65 -> 98.37 | Improved |
| use normalized step size in p-GD | p-GD p=1.5, conv net · MNIST test | test accuracy (%): 98.68 -> 99.19 | Improved |
| use p=3 instead of p=2 (SGD) in p-GD | ResNet-18 · CIFAR-10 test | test accuracy (%): 93.90 -> 94.01 | Improved |
| use p=3 instead of p=2 (SGD) in p-GD | VGG-11 · CIFAR-10 test | test accuracy (%): 90.15 -> 90.85 | Improved |
| use p=3 instead of p=2 (SGD) in p-GD | MobileNet-v2 · CIFAR-10 test | test accuracy (%): 91.97 -> 93.23 | Improved |
| use p=3 instead of p=2 (SGD) in p-GD | RegNetX-200mf · CIFAR-10 test | test accuracy (%): 92.75 -> 94.07 | Improved |
| use sparse-inducing p=1.1 instead of p=2 (SGD) in p-GD | VGG-11 · CIFAR-10 test | test accuracy (%): 90.15 -> 88.19 | Hurt |
| use p=1.1 instead of p=2 (SGD) in p-GD | ResNet-18 · CIFAR-10 test | test accuracy (%): 93.90 -> 92.63 | Hurt |
| use p=10 instead of p=3 in p-GD | VGG-11 · CIFAR-10 test | test accuracy (%): 90.85 -> 88.78 | Hurt |
| use p=10 instead of p=3 in p-GD | RegNetX-200mf · CIFAR-10 test | test accuracy (%): 94.07 -> 92.97 | Hurt |
| run p-GD with p=1.1 instead of p=2 | linear classifier, 15 points in R^100 · l_1.1 norm of rescaled classifier | norm size: 6.161 -> 5.477 | Improved |
| run p-GD with p=10 instead of p=2 | linear classifier, 15 points in R^100 · l_10 norm of rescaled classifier | norm size: 0.382 -> 0.280 | Improved |
Closest papers by meaning
- Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently (NeurIPS 2022 · similarity 0.92)
- Mask in the Mirror: Implicit Sparsification (ICLR 2025 · similarity 0.91)
- Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization (ICLR 2019 · similarity 0.90)
- Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias? (ICML 2025 · similarity 0.89)
- Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent (ICML 2025 · similarity 0.88)
- Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
