AI Paper Atlas

Research map / Optimization / Optimizers

Reparameterizing Mirror Descent as Gradient Descent

Ehsan Amid, Manfred K. Warmuth

NeurIPS 2020 · Optimization › Optimizers · approach: Constrained and non-backprop training · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Apply EGU± reparameterization to the CNN's final-layer weightsCNN · MNIST final-layer weightsL1 norm: 571.1 -> 176.7Improved
Apply EGU± reparameterization to the CNN's final-layer weightsCNN · MNIST testtop-1 accuracy: 99.13% -> 99.02Hurt
Clamp final-layer weights below 1e-3 to zeroReparameterized EGU± CNN · MNIST testtop-1 accuracy: 99.02 -> 99.02No clear effect
Clamp final-layer weights below 5e-3 to zeroReparameterized EGU± CNN · MNIST testtop-1 accuracy: 99.02 -> 99.00Hurt
Clamp final-layer weights below 1e-2 to zeroReparameterized EGU± CNN · MNIST testtop-1 accuracy: 99.00 -> 98.96Hurt
Clamp final-layer weights below 5e-2 to zeroReparameterized EGU± CNN · MNIST testtop-1 accuracy: 98.96 -> 98.20Hurt
Clamp final-layer weights below 1e-1 to zeroReparameterized EGU± CNN · MNIST testtop-1 accuracy: 98.20 -> 97.48Hurt
Threshold final-layer weights to zeroReparameterized EGU± CNN · MNIST final-layer weightssparsity: 00.00 -> 98.90Improved
Use tempered EGU± with temperature τ for underdetermined linear regressionUnderdetermined linear regression · Underdetermined linear regression, N<dsolution normImproved
Reparameterize the continuous updates as gradient descentGD, EGU±, and tempered EGU± on underdetermined linear regression · Underdetermined linear regressionsolution trajectory agreementNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy