Research map / Optimization / Optimizers
Training Neural Networks Without Gradients: A Scalable ADMM Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ADMM instead of SGD | Neural network training · SVHN | time to 95% test accuracy: 28.3 seconds -> 13.3 seconds | Helped |
| Use ADMM instead of L-BFGS | Neural network training · SVHN | time to 95% test accuracy: 3.3 seconds -> 13.3 seconds | Hurt |
| Use ADMM instead of conjugate gradients | Neural network training · SVHN | time to 95% test accuracy: 10.1 seconds -> 13.3 seconds | Hurt |
| Use ADMM instead of L-BFGS | Neural network training · Higgs | time to 64% test accuracy: 181 seconds -> 7.8 seconds | Helped |
| Use ADMM instead of conjugate gradients | Neural network training · Higgs | time to 64% test accuracy: 44 minutes -> 7.8 seconds | Helped |
| Use ADMM instead of SGD | Neural network training · Higgs | reaching 64% test accuracy | Helped |
| Use L-BFGS instead of ADMM | Neural network training · Higgs | test accuracy: 75% | Helped |
Closest papers by meaning
- Differentiable Optimization of Generalized Nondecomposable Functions using Linear Programs (NeurIPS 2021 · similarity 0.86)
- Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.86)
- A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks (NeurIPS 2024 · similarity 0.85)
- Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC (ICML 2024 · similarity 0.85)
- Towards a Better Theoretical Understanding of Independent Subnetwork Training (ICML 2024 · similarity 0.85)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.