Research map / Bandits and online learning / Online convex optimization and games
A new regret analysis for Adam-type algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use constant β1 instead of a schedule decreasing to zero | AMSGrad · Online convex optimization | regret: O(√T) | Helped |
| Use constant β1 instead of a schedule decreasing to zero | AdamNc · Online convex optimization | regret: O(√T) | Helped |
| Use constant β1 instead of a schedule decreasing to zero | Sadam · Strongly convex online optimization | regret: O(log T) | Helped |
| Use constant β1 instead of β1t = β1/t | ZO-AdaMM · Convex zeroth-order optimization | expected regret: O(√T) | Helped |
| Use the new analysis instead of the prior nonconvex AMSGrad proof | AMSGrad · Nonconvex stochastic optimization | average squared gradient norm: slightly better constants | Helped |
| Use a nonincreasing variable β1 schedule rather than constant β1 | Adam-type methods · Online convex optimization | regret-bound constants: 1/(1−β1)^2 | Hurt |
Closest papers by meaning
- Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer (ICML 2026 · similarity 0.90)
- An Equivalence Between Static and Dynamic Regret Minimization (NeurIPS 2024 · similarity 0.89)
- Dynamic Regret Reduces to Kernelized Static Regret (NeurIPS 2025 · similarity 0.88)
- Small-loss Adaptive Regret for Online Convex Optimization (ICML 2024 · similarity 0.88)
- A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD (ICML 2025 · similarity 0.88)
- Dynamic Regret in Online Convex Optimization with Indicator Switching Costs (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.