Research map / Optimization / Nonconvex and smooth optimization
High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use burn-in instead of no warm-up or burn-in (ablation) | Normalized SGD with clipping and momentum · BERT-Large masked language modeling pretraining | eval accuracy | Helped |
| Use a sufficiently long burn-in period instead of a short burn-in period (ablation) | Normalized SGD with clipping and momentum · BERT-Large masked language modeling pretraining | eval accuracy | Helped |
| Use burn-in instead of linear warm-up (ablation) | Normalized SGD with clipping and momentum · BERT-Large masked language modeling pretraining | eval accuracy: 62.95% vs. 63.45% | Hurt |
| Replace linear warm-up with burn-in (ablation) | LAMB · BERT-Large masked language modeling pretraining | eval accuracy | Hurt |
| Use neither warm-up nor burn-in (ablation) | LAMB · BERT-Large masked language modeling pretraining | eval accuracy | Hurt |
Closest papers by meaning
- High-Probability Bound for Non-Smooth Non-Convex Stochastic Optimization with Heavy Tails (ICML 2024 · similarity 0.92)
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.91)
- High-probability complexity bounds for stochastic non-convex minimax optimization (NeurIPS 2024 · similarity 0.91)
- Private Stochastic Convex Optimization with Heavy Tails: Near-Optimality from Simple Reductions (NeurIPS 2024 · similarity 0.91)
- Non-Asymptotic and Non-Lipschitzian Bounds on Optimal Values in Stochastic Optimization Under Heavy Tails (ICML 2025 · similarity 0.90)
- Generalization Bounds for Heavy-Tailed SDEs through the Fractional Fokker-Planck Equation (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.