Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Thompson sampling analysis
Analyze and improve Thompson sampling for combinatorial and high-dimensional contextual bandits.
84 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Contextual bandits and Thompson sampling, keyword cluster “thompson sampling · bayesian · ts”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
24 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 16.1% to 11.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Double Thompson Sampling for Dueling Bandits (NeurIPS 2016 · 40 citations)
- 2017 · Embedded Bandits for Large-Scale Black-Box Optimization (AAAI 2017 · 2 citations)
- 2018 · Improved Regret Bounds for Thompson Sampling in Linear Quadratic Control Problems (ICML 2018 · 52 citations)
- 2019 · Thompson Sampling for Multinomial Logit Contextual Bandits (NeurIPS 2019 · 23 citations)
- 2020 · Latent Bandits Revisited (NeurIPS 2020 · 19 citations)
- 2021 · Neural Thompson Sampling (ICLR 2021 · 31 citations)
- 2022 · A Unifying Theory of Thompson Sampling for Continuous Risk-Averse Bandits (AAAI 2022 · 12 citations)
- 2023 · Meta-Learning for Simple Regret Minimization (AAAI 2023 · 3 citations)
- 2024 · The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models (AAAI 2024 · 2 citations)
- 2025 · Constrained Linear Thompson Sampling (NeurIPS 2025 · 1 citation)
- 2026 · Prior Diffusiveness and Regret in the Linear-Gaussian Bandit (ICML 2026 · 0 citations)
Earliest papers here
- Double Thompson Sampling for Dueling Bandits (NeurIPS 2016 · 40 citations)
- Embedded Bandits for Large-Scale Black-Box Optimization (AAAI 2017 · 2 citations)
- Improved Regret Bounds for Thompson Sampling in Linear Quadratic Control Problems (ICML 2018 · 52 citations)
What has been tried
71 papers here report 466 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 11%, hurt 20%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add adaptive scale truncation | Uniform bandits · asymptotic regret optimality | Helped | The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models (AAAI 2024 · 2 citations) | |
| Increase r-COLTS resampling from 1 to 3 samples per round | r-COLTS simulation, T = 5 \cdot 10^4, 100 trials · regret | 658 ± 170 -> 301 ± 102 | Helped | Constrained Linear Thompson Sampling (NeurIPS 2025 · 1 citation) |
| Use VITS-II instead of LMC-TS | Quadratic bandit, ζ=0.1 and ζ=1 · cumulative regret | Helped | $\mathtt{VITS}$ : Variational Inference Thompson Sampling for contextual bandits (ICML 2024 · 1 citation) | |
| Increase the number of tasks m | Synthetic linear bandit · regret | Hurt | Improved Bayes Regret Bounds for Multi-Task Hierarchical Bayesian Bandit Algorithms (NeurIPS 2024 · 1 citation) | |
| Use a clipped Beta distribution rather than a standard Beta distribution for sampling the binary component | Zero-inflated MAB · overestimation probability for suboptimal arms | Helped | Zero-Inflated Bandits (ICML 2025 · 0 citations) | |
| Increase the trade-off parameter α | Five-arm Bernoulli bandit · GDP parameter | Helped | Connecting Thompson Sampling and UCB: Towards More Efficient Trade-offs Between Privacy and Regret (ICML 2025 · 0 citations) | |
| Add optimistic sampling (use M=O(log L) rather than M=1) | Multi-objective linear contextual bandits · Pareto regret | Helped | Thompson Sampling for Multi-Objective Linear Contextual Bandit (NeurIPS 2025 · 0 citations) | |
| Increase ensemble size M | Nonlinear bandits · regret | Helped | Scalable Exploration via Ensemble++ (NeurIPS 2025 · 0 citations) | |
| choose c bounded away from 0 rather than near 0 | Synthetic linear bandits, sparse noise · regret | Helped | Variance-Aware Feel-Good Thompson Sampling for Contextual Bandits (NeurIPS 2025 · 0 citations) | |
| Use a feel-good bonus λ=0.1 | Linear-20d (β=10³) · cumulative regret | 62.6 ± 9.5 -> 50.8 ± 15.8 | Helped | Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown (NeurIPS 2025 · 0 citations) |
| Use a treatment embedding correlated with the true reward-relevant dimension | Treatment embedding misspecification · regret | Helped | Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions (NeurIPS 2025 · 0 citations) | |
| Vary the Feel-Good exploration scale α | Synthetic data, d = 5, 10, 15 · cumulative regret | No clear effect | Feel-Good Thompson Sampling for Contextual Dueling Bandits (ICML 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- Greedy · 3 of 12 papers
- Epsilon-greedy · 3 of 12 papers
- LinUCB · 3 of 12 papers
- SAFE-LTS · 2 of 12 papers
- LMC-TS · 2 of 12 papers
- LinTS · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 39 papers; from OpenAlex and the papers' affiliations).
- Amazon · 6 papers
- Seoul National University · 4 papers
- Google · 3 papers
- ByteDance · 3 papers
- University of Southern California · 2 papers
- RIKEN AIP · 2 papers
- The University of Tokyo · 2 papers
- Yale University · 2 papers
- University of Michigan - Ann Arbor · 2 papers
- Google DeepMind · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Gaussian process bandit regret (26 papers)
Elsewhere in the topic
- Oracle-efficient contextual bandits (168 papers)
- Linear and generalized linear bandits (88 papers)
- Practical contextual bandit extensions (77 papers)
- Causal and structured bandits (61 papers)
- Ranking bandits (42 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).