AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

On ADMM in Deep Learning: Convergence and Saturation-Avoidance

Jinshan Zeng, Shao-Bo Lin, Yuan Yao, Ding-Xuan Zhou

JMLR 2021 · Optimization › Implicit bias and generalization · approach: Shallow ReLU network convergence · 22 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use ADMM instead of SGD to train sigmoid netsSGD (sigmoid) · Approximation of x^2approximation error: 2.46e-4 -> 2.53e-9Improved
use ADMM instead of SGD to train sigmoid netsSGD (sigmoid) · Approximation of product gate uvapproximation error: 1.13e-3 -> 2.62e-9Improved
use ADMM instead of SGD to train sigmoid netsSGD (sigmoid) · Learning L1 radial function (noise 0.1)test error: 4.58e-5 -> 1.69e-5Improved
use ADMM instead of SGD to train sigmoid netsSGD (sigmoid) · Learning L2 radial function (noise 0.1)test error: 9.33e-5 -> 9.28e-6Improved
use sigmoid-ADMM instead of ReLU AdamAdam (ReLU) · Approximation of x^2approximation error: 3.33e-8 -> 2.53e-9Improved
use sigmoid-ADMM instead of ReLU SGDMSGDM (ReLU) · Approximation of product gate uvapproximation error: 3.37e-7 -> 2.62e-9Improved
use sigmoid-ADMM instead of ReLU AdamAdam (ReLU) · Learning L1 radial function (noise 0.1)test error: 2.16e-5 -> 1.69e-5Improved
use sigmoid-ADMM instead of ReLU AdamAdam (ReLU) · Learning L2 radial function (noise 0.1)test error: 1.02e-5 -> 9.28e-6Improved
use sigmoid-ADMM instead of ReLU Adam (run time)Adam (ReLU) · Approximation of x^2run time (s): 38.26 -> 9.47Improved
use sigmoid-ADMM instead of ReLU SGD (run time)SGD (ReLU) · Learning L2 radial function (noise 0.1)run time (s): 104.52 -> 47.36Improved
vary ADMM penalty beta, choosing beta=1 (ablation)ADMM (sigmoid) · Approximation of x^2approximation errorImproved
vary initialization scheme (MSRA, orthogonal, LeCun, Xavier) (ablation)ADMM (sigmoid) · Approximation of product gate uvapproximation errorNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy