Research map / Optimization / Optimizers
Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · BN50 | loss: 2.036 -> 1.916 | Helped |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · BN50 | WER: 17.4 -> 16.4 | Helped |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · SWB300 | loss: 1.626 -> 1.551 | Helped |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · SWB300 | SWB WER: 10.3 -> 10.0 | Helped |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · SWB300 | CH WER: 18.3 -> 18.2 | No clear effect |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · CIFAR10 | error rate: 8.49 -> 7.52 | Helped |
| Add evolutionary interaction to the population optimizer (ablation) | ESGD · PTB | test perplexity: 64.58 -> 63.73 | Helped |
| Use a randomized mixture of ADAM and SGD optimizers | ESGD on CIFAR10 · CIFAR10 | error rate | Hurt |
| Add the ESGD population evolution process | Population baseline · BN50 | loss: 2.029 -> 1.916 | Helped |
| Add the ESGD population evolution process | Population baseline · CIFAR10 | error rate: 8.24 -> 7.52 | Helped |
Closest papers by meaning
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (NeurIPS 2019 · similarity 0.86)
- Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise (ICLR 2025 · similarity 0.86)
- Fast and Accurate Stochastic Gradient Estimation (NeurIPS 2019 · similarity 0.86)
- Guided evolutionary strategies: augmenting random search with surrogate gradients (ICML 2019 · similarity 0.86)
- A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks (NeurIPS 2024 · similarity 0.85)
- Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.