Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Oracle-efficient contextual bandits
Build practical contextual bandit algorithms using regression oracles and handling misspecification or privacy.
168 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Contextual bandits and Thompson sampling, keyword cluster “contextual bandits · contexts · user”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
43 papers in 2023–24 and 41 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 28.9% to 30.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · BISTRO: An Efficient Relaxation-Based Method for Contextual Bandits (ICML 2016 · 69 citations)
- 2017 · Provably Optimal Algorithms for Generalized Linear Contextual Bandits (ICML 2017 · 144 citations)
- 2018 · Online Learning with an Unknown Fairness Metric (NeurIPS 2018 · 78 citations)
- 2019 · Balanced Linear Contextual Bandits (AAAI 2019 · 41 citations)
- 2020 · Neural Contextual Bandits with UCB-based Exploration (ICML 2020 · 54 citations)
- 2021 · Multinomial Logit Contextual Bandits: Provable Optimality and Practicality (AAAI 2021 · 16 citations)
- 2022 · Neural Contextual Bandits with Deep Representation and Shallow Exploration (ICLR 2022 · 18 citations)
- 2023 · Stochastic Contextual Bandits with Long Horizon Rewards (AAAI 2023 · 4 citations)
- 2024 · Risk-Aware Continuous Control with Neural Contextual Bandits (AAAI 2024 · 4 citations)
- 2025 · Neural Combinatorial Clustered Bandits for Recommendation Systems (AAAI 2025 · 3 citations)
- 2026 · Contextual Slate GLM Bandits with Limited Adaptivity (ICML 2026 · 0 citations)
Earliest papers here
- BISTRO: An Efficient Relaxation-Based Method for Contextual Bandits (ICML 2016 · 69 citations)
- Linear Contextual Bandits with Knapsacks (NeurIPS 2016 · 60 citations)
- Efficient Algorithms for Adversarial Contextual Learning (ICML 2016 · 39 citations)
What has been tried
126 papers here report 739 measured changes (a component added, removed or swapped, compared with a baseline): helped 69%, no clear effect 9%, hurt 16%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase risk level α from 0.5 to 0.995 | Synthetic environment · reward | Hurt | Risk-Aware Continuous Control with Neural Contextual Bandits (AAAI 2024 · 4 citations) | |
| set λ from 0.01 to 0.001 | MNIST (C=2000) · cumulative regret | 2501 ±93 -> 1989 ±67 | Helped | Robust Neural Contextual Bandit against Adversarial Corruptions (NeurIPS 2024 · 2 citations) |
| Estimate D and σ² instead of using their true values | Random intercept, D>0 · cumulative regret | No clear effect | Mixed-Effects Contextual Bandits (AAAI 2024 · 1 citation) | |
| Add local-derivative estimation instead of using the uniform lower bound $\kappa$ | Contextual Dueling Bandits with sigmoid link function · regret | \tilde O\big({dB^{1.5}\sqrt{T}} + {dBC}/{\kappa} + d^{2}B^{2}(1/\kappa^2 + B/\kappa)\big) | Helped | Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback (ICML 2025 · 0 citations) |
| Vary the hyperparameter τ across reasonable values | Experiments across settings · frequentist regret | No clear effect | Leveraging Offline Data in Linear Latent Contextual Bandits (ICML 2025 · 0 citations) | |
| use the correct diffusion prior but a linear-Gaussian reward model instead of the nonlinear reward model | Synthetic contextual bandits with nonlinear rewards · cumulative regret | Helped | Diffusion Models Meet Contextual Bandits (NeurIPS 2025 · 0 citations) | |
| add the safety condition | OpenML multiclass classification datasets · percentage of constraints violated | Helped | Conservative Contextual Bandits: Beyond Linear Representations (ICLR 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 28 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MovieLens 25M · 2 of 28 papers
- MovieLens-1M · 2 of 28 papers
Baselines
- LinUCB · 4 of 28 papers
- LinTS · 4 of 28 papers
- epsilon-greedy · 3 of 28 papers
- TS · 2 of 28 papers
- NeuralUCB · 2 of 28 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 84 papers; from OpenAlex and the papers' affiliations).
- Massachusetts Institute of Technology · 9 papers
- Seoul National University · 8 papers
- University of Virginia · 6 papers
- University of Illinois, Urbana Champaign · 5 papers
- University of Southern California · 4 papers
- Google · 4 papers
- Columbia University · 3 papers
- University of California, Los Angeles · 3 papers
- Princeton University · 2 papers
- Meta · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Linear and generalized linear bandits (88 papers)
- Practical contextual bandit extensions (77 papers)
- Causal and structured bandits (61 papers)
- Ranking bandits (42 papers)
Elsewhere in the topic
- Thompson sampling analysis (84 papers)
- Gaussian process bandit regret (26 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).