Research map / Optimization / Nonconvex and smooth optimization
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning
Kuangyu Ding, Jingyang Li, Kim-Chuan Toh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MSBPG instead of AdamW | ViT Tiny · CIFAR-100 | test accuracy: 58.77 -> 62.37 | Improved |
| use MSBPG instead of SGD | ViT Small · CIFAR-100 | test accuracy: 61.90 -> 63.59 | Improved |
| use MSBPG instead of SGD | ConvNext Atto · CIFAR-100 | test accuracy: 74.46 -> 75.98 | Improved |
| use MSBPG instead of SGD | ConvNext Femto · CIFAR-100 | test accuracy: 75.96 -> 76.47 | Improved |
| use MSBPG instead of AdamW | Transformer-XL · WikiText-103 | test perplexity: 32.17 -> 31.07 | Improved |
| use MSBPG instead of SGD | Transformer-XL · WikiText-103 | test perplexity: 33.81 -> 31.07 | Improved |
| use Bregman polynomial kernel with implicit update instead of Euclidean update | MSBPG · training with large stepsize or initial point | robustness to learning rate | Improved |
Closest papers by meaning
- A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems (AAAI 2024 · similarity 0.91)
- Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reduction (ICML 2021 · similarity 0.90)
- ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization (JMLR 2020 · similarity 0.89)
- A Hybrid Stochastic-Deterministic Minibatch Proximal Gradient Method for Efficient Optimization and Generalization (TPAMI 2021 · similarity 0.89)
- Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization (JMLR 2022 · similarity 0.88)
- Adaptive Methods for Nonconvex Optimization (NeurIPS 2018 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
