Research map / Optimization / Optimizers
Stochastic Block BFGS: Squeezing More Curvature out of Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use previous search directions as the sketch (prev variant) instead of Gaussian or factored sketches | stochastic block BFGS with SVRG · logistic regression on LIBSVM datasets (Figures 1-2) | data passes and time to reach low error | Helped |
| Use previous search directions as the sketch instead of Gaussian or factored sketches | stochastic block BFGS with SVRG · url-combined | time to reach low error | Hurt |
| Replace classical L-BFGS metric with stochastic block BFGS (block sketched curvature) combined with SVRG | SVRG · logistic regression benchmarks (Figures 1-2) | data passes to reach low error | Helped |
| Use the stochastic block BFGS metric on problems with d much smaller than n (HIGGS, SUSY) | SVRG with curvature · HIGGS, SUSY | error vs. SVRG | Helped |
| Use factored-form self-conditioning sketch (fact) instead of Gaussian sketch (ablation) | stochastic block BFGS with SVRG · logistic regression benchmarks | efficiency | No clear effect |
| Replace classical L-BFGS (Moritz et al. MNJ) with block limited-memory BFGS | SVRG with curvature · url-combined | time to reach low error | Mixed |
Closest papers by meaning
- Structured BFGS Method for Optimal Doubly Stochastic Matrix Approximation (AAAI 2023 · similarity 0.91)
- Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods (CVPR 2021 · similarity 0.90)
- Practical Quasi-Newton Methods for Training Deep Neural Networks (NeurIPS 2020 · similarity 0.88)
- Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reduction (ICML 2021 · similarity 0.88)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.86)
- Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.