Research map / Bandits and online learning / Multi-armed bandits
Adam with Bandit Sampling for Deep Learning
Rui Liu, Tianyi Wu, Barzan Mozafari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add bandit-based example sampling to Adam (Adambs) instead of uniform sampling | Adam · CNN on CIFAR10 and CIFAR100 | training loss vs wall-clock time | Improved |
| add bandit-based example sampling to Adam (Adambs) | Adam · CNN on Fashion MNIST | training loss vs wall-clock time | Mixed |
| add bandit-based example sampling to Adam (Adambs) | Adam · LSTM on MNIST and Fashion MNIST | training loss vs wall-clock time | Improved |
| add bandit-based example sampling to Adam (Adambs) | Adam · RCNN on IMDB | training loss vs wall-clock time | Mixed |
| bandit sampling versus importance sampling on Adam (Adam-impt) | Adam · CNN, RNN and RCNN benchmarks | training loss vs wall-clock time | Improved |
| use Adam-based methods instead of basic SGD variants (SGD, Momentum, Adagrad, RMSprop) | Adambs comparison · appendix CNN experiments | training loss | Improved |
| bandit sampling over individual examples versus Adam with adaptive or uniform sampling over prefixed batches | Adam · doubly heavy-tailed gradient distribution | convergence rate bound | Improved |
| add bandit sampling when the d sqrt(T) term dominates the bound | Adam · theoretical regret bound | convergence rate | No clear effect |
Closest papers by meaning
- ADAM Optimization with Adaptive Batch Selection (ICLR 2025 · similarity 0.94)
- A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD (ICML 2025 · similarity 0.89)
- Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks (NeurIPS 2025 · similarity 0.87)
- Prodigy: An Expeditiously Adaptive Parameter-Free Learner (ICML 2024 · similarity 0.87)
- ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate (NeurIPS 2024 · similarity 0.87)
- Adam-mini: Use Fewer Learning Rates To Gain More (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
