Research map / Optimization / Optimizers
Reparameterizing Mirror Descent as Gradient Descent
Ehsan Amid, Manfred K. Warmuth
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply EGU± reparameterization to the CNN's final-layer weights | CNN · MNIST final-layer weights | L1 norm: 571.1 -> 176.7 | Improved |
| Apply EGU± reparameterization to the CNN's final-layer weights | CNN · MNIST test | top-1 accuracy: 99.13% -> 99.02 | Hurt |
| Clamp final-layer weights below 1e-3 to zero | Reparameterized EGU± CNN · MNIST test | top-1 accuracy: 99.02 -> 99.02 | No clear effect |
| Clamp final-layer weights below 5e-3 to zero | Reparameterized EGU± CNN · MNIST test | top-1 accuracy: 99.02 -> 99.00 | Hurt |
| Clamp final-layer weights below 1e-2 to zero | Reparameterized EGU± CNN · MNIST test | top-1 accuracy: 99.00 -> 98.96 | Hurt |
| Clamp final-layer weights below 5e-2 to zero | Reparameterized EGU± CNN · MNIST test | top-1 accuracy: 98.96 -> 98.20 | Hurt |
| Clamp final-layer weights below 1e-1 to zero | Reparameterized EGU± CNN · MNIST test | top-1 accuracy: 98.20 -> 97.48 | Hurt |
| Threshold final-layer weights to zero | Reparameterized EGU± CNN · MNIST final-layer weights | sparsity: 00.00 -> 98.90 | Improved |
| Use tempered EGU± with temperature τ for underdetermined linear regression | Underdetermined linear regression · Underdetermined linear regression, N<d | solution norm | Improved |
| Reparameterize the continuous updates as gradient descent | GD, EGU±, and tempered EGU± on underdetermined linear regression · Underdetermined linear regression | solution trajectory agreement | No clear effect |
Closest papers by meaning
- Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently (NeurIPS 2022 · similarity 0.88)
- A Unified Approach to Controlling Implicit Regularization via Mirror Descent (JMLR 2023 · similarity 0.87)
- The Statistical Complexity of Early-Stopped Mirror Descent (NeurIPS 2020 · similarity 0.87)
- Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity (NeurIPS 2026 · similarity 0.86)
- RepAn: Enhanced Annealing through Re-parameterization (CVPR 2024 · similarity 0.85)
- Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
