Research map / Bandits and online learning / Online convex optimization and games
Online decision-making with feedback structure
Analyze online prediction under delayed, bandit, strategic, or dynamical feedback.
81 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Online convex optimization and games, keyword cluster “price · pricing · auctions”
State History Tried Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
24 papers in 2023–24 and 27 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 10.5% to 9.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Online Instrumental Variable Regression with Applications to Online Linear System Identification (AAAI 2016 · 11 citations)
- 2017 · Online Prediction with Selfish Experts (NeurIPS 2017 · 3 citations)
- 2018 · Generalized Inverse Optimization through Online Learning (NeurIPS 2018 · 33 citations)
- 2019 · Online Meta-Learning (ICML 2019 · 93 citations)
- 2020 · Online Bayesian Persuasion (NeurIPS 2020 · 22 citations)
- 2021 · Multi-Receiver Online Bayesian Persuasion (ICML 2021 · 6 citations)
- 2022 · Simultaneously Learning Stochastic and Adversarial Bandits under the Position-Based Model (AAAI 2022 · 3 citations)
- 2023 · CLIP-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments (NeurIPS 2023 · 3 citations)
- 2024 · Regret Analysis of Repeated Delegated Choice (AAAI 2024 · 2 citations)
- 2025 · Online Learning of Coalition Structures by Selfish Agents (AAAI 2025 · 2 citations)
- 2026 · When Drafts Evolve: Speculative Decoding Meets Online Learning (ICML 2026 · 0 citations)
Earliest papers here
- Online Instrumental Variable Regression with Applications to Online Linear System Identification (AAAI 2016 · 11 citations)
- A Bandit Framework for Strategic Regression (NeurIPS 2016 · 6 citations)
- Online Prediction with Selfish Experts (NeurIPS 2017 · 3 citations)
What has been tried
62 papers here report 396 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 9%, hurt 20%, mixed 8%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| increase re-solving frequency | Input I and Input II, T from 10^3 to 10^6 · regret | Helped | Wait-Less Offline Tuning and Re-solving for Online Decision Making (ICML 2025 · 0 citations) | |
| Replace no exploration (M0) with M2's exploration and learning restart | T=105 · regret | 123.03 -> 24.00 | Helped | Decoupling Learning and Decision-Making: Breaking the $\mathcal{O}(\sqrt{T})$ Barrier in Online Resource Allocation with First-Order Methods (ICML 2024 · 0 citations) |
| epsilon-Greedy exploration parameter m below 500 (m=30 to 400) | 5-arm toy bandit · mean regret | Hurt | Incentivized Learning in Principal-Agent Bandit Games (ICML 2024 · 0 citations) | |
| Add a tightness parameter in the dual update | Constrained causal logistic bandits · cumulative constraint violations | eO(nS√T + n²S²κZ + (1 + ϵ)²√T − ϵT) -> 0 | Helped | Causal Logistic Bandits with Counterfactual Fairness Constraints (ICML 2025) |
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 51 papers; from OpenAlex and the papers' affiliations).
- Polytechnic Institute of Milan · 4 papers
- Politecnico di Milano · 4 papers
- University of California, Berkeley · 4 papers
- University of Pennsylvania · 3 papers
- Columbia University · 3 papers
- Bocconi University · 3 papers
- National University of Singapore · 2 papers
- Stanford University · 2 papers
- Google DeepMind · 2 papers
- Meta · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Online learning with feedback graphs (155 papers)
- Best-of-both-worlds bandit algorithms (125 papers)
- Learning for pricing and auctions (101 papers)
- Online allocation with constraints (77 papers)
- Online linear-quadratic control regret (36 papers)
- Learning-augmented online algorithms (73 papers)
Elsewhere in the topic
- Constrained and unconstrained OCO algorithms (85 papers)
- Last-iterate convergence in games (65 papers)
- Dynamic and adaptive regret (57 papers)
- Parameter-free non-stationary online learning (45 papers)
- Fast no-regret dynamics in games (37 papers)
- Counterfactual regret minimization (15 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).