Research map / Optimization / Optimizers
A Multi-Batch L-BFGS Method for Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Robust (overlap-based) gradient differences for quasi-Newton updating instead of inconsistent gradient differences (ablation) | multi-batch L-BFGS · webspam logistic regression | stability of gradient norm across 10 seeds | Helped |
| Use multi-batch L-BFGS instead of multi-batch gradient descent (H_k = I) | multi-batch gradient method · webspam logistic regression | convergence of gradient norm | Helped |
| Overlap size varied from 5% to 30% of batch (ablation) | robust multi-batch L-BFGS · webspam logistic regression | gradient norm convergence | No clear effect |
| Robust L-BFGS compared with serial SGD at r=1% | robust multi-batch L-BFGS · webspam logistic regression | accuracy and rate per epoch | No clear effect |
| Robust sample-consistent overlap updating under node failures with p in {0.1,0.3,0.5} (ablation) | robust multi-batch L-BFGS · webspam with node failures | stability over 10 runs | Helped |
| Increase number of MPI processes (strong scaling) for gradient computation | robust multi-batch L-BFGS · artificial data d=1e4, n=1e7, K up to 128 | gradient computation time | Helped |
| Use a larger step length alpha=1 instead of 0.1 (ablation) | robust multi-batch L-BFGS · logistic regression benchmark datasets | convergence of gradient norm | Helped |
Closest papers by meaning
- A Progressive Batching L-BFGS Method for Machine Learning (ICML 2018 · similarity 0.92)
- LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning (NeurIPS 2018 · similarity 0.88)
- Bayesian Distributed Stochastic Gradient Descent (NeurIPS 2018 · similarity 0.87)
- A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent (ICML 2019 · similarity 0.87)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.86)
- Faster Stochastic Optimization with Arbitrary Delays via Adaptive Asynchronous Mini-Batching (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.