Research map / Optimization / Optimizers
A Progressive Batching L-BFGS Method for Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-batch curvature pairs with 25% sample overlap instead of full overlap (ablation) | PBQN · CIFAR-10 ConvNet | test accuracy: 62.46 -> 67.37 | Helped |
| Use multi-batch curvature pairs with 25% sample overlap instead of full overlap (ablation) | PBQN · MNIST AlexNet-like network | test accuracy: 99.05 -> 99.16 | Helped |
| Use multi-batch curvature pairs with 25% sample overlap instead of full overlap (ablation) | PBQN · CIFAR-10 AlexNet-like network | test accuracy: 72.74 -> 73.02 | Helped |
| Use multi-batch curvature pairs with 25% sample overlap instead of full overlap (ablation) | PBQN · CIFAR-10 ResNet18 | test accuracy: 69.44 -> 70.28 | Helped |
| Use progressive-batching L-BFGS rather than SG or Adam | Neural network training · CIFAR-10 ConvNet, MNIST AlexNet-like network, CIFAR-10 AlexNet-like network, and CIFAR-10 ResNet18 | test accuracy | Mixed |
| Use progressive batching with L-BFGS rather than SG or SVRG | Logistic regression · spam and covertype | training error | Mixed |
Closest papers by meaning
- A Multi-Batch L-BFGS Method for Machine Learning (NeurIPS 2016 · similarity 0.92)
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (ICLR 2017 · similarity 0.87)
- Practical Quasi-Newton Methods for Training Deep Neural Networks (NeurIPS 2020 · similarity 0.87)
- Stochastic Block BFGS: Squeezing More Curvature out of Data (ICML 2016 · similarity 0.86)
- A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent (ICML 2019 · similarity 0.86)
- From Logistic Regression to the Perceptron Algorithm: Exploring Gradient Descent with Large Step Sizes (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.