Research map / Bandits and online learning
Contextual bandits and Thompson sampling: research map
567 papers on Contextual bandits and Thompson sampling in Bandits and online learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 7 approaches. The busiest year so far is 2023.
What it is
In a contextual bandit the learner sees side information (a user, a query) before choosing an action, as in recommendation. Thompson sampling is a popular Bayesian strategy: sample a plausible model of the world and act as if it were true.
Terms you will see on this page:
- Context
- the features observed before each decision.
- Linear bandit
- a bandit whose expected reward is linear in the features of the action.
- Thompson sampling
- choosing the action that is best under a model sampled from the posterior.
Start reading with the most cited papers here:
- Fairness in Learning: Classic and Contextual Bandits (NeurIPS 2016 · 191 citations)
- Provably Optimal Algorithms for Generalized Linear Contextual Bandits (ICML 2017 · 144 citations)
Within Bandits and online learning, its share shrank from 26.8% in 2023–24 to 22.4% in 2025–26 (149 → 133 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Contextual bandits and Thompson sampling in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Ranking bandits rising: 5% → 9% of the topic (7 → 12 papers)
- Oracle-efficient contextual bandits rising: 29% → 31% of the topic (43 → 41 papers)
- Thompson sampling analysis falling: 16% → 11% of the topic (24 → 15 papers)
- Causal and structured bandits falling: 13% → 11% of the topic (19 → 14 papers)
Papers from 2025–2026 that are already cited most:
- Neural Combinatorial Clustered Bandits for Recommendation Systems (AAAI 2025 · 3 citations)
- Improved Regret Bounds for Gaussian Process Upper Confidence Bound in Bayesian Optimization (NeurIPS 2025 · 0 citations)
Approaches and key papers
Contextual bandits · 452 papers
Keywords: contextual bandits, contexts, user
Approaches in this cluster:
- Oracle-efficient contextual bandits (175 papers)
Build practical contextual bandit algorithms using regression oracles and handling misspecification or privacy. - Linear and generalized linear bandits (90 papers)
Derive improved regret bounds for stochastic linear bandits with efficient updates. - Practical contextual bandit extensions (80 papers)
Tune, infer from, cluster, or meta-learn contextual bandit algorithms. - Causal and structured bandits (61 papers)
Exploit causal graphs and auxiliary feedback to learn good interventions. - Ranking bandits (46 papers)
Learn to rank online from click feedback with unimodal and semi-bandit methods.
Most cited and most cited since 2024:
- Fairness in Learning: Classic and Contextual Bandits (NeurIPS 2016 · 191 citations)
- Provably Optimal Algorithms for Generalized Linear Contextual Bandits (ICML 2017 · 144 citations)
- Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health (AAAI 2024 · 11 citations)
- Causal Bandits for Linear Structural Equation Models (NeurIPS 2024 · 5 citations)
Thompson sampling · 115 papers
Keywords: thompson sampling, bayesian, ts
Approaches in this cluster:
- Thompson sampling analysis (89 papers)
Analyze and improve Thompson sampling for combinatorial and high-dimensional contextual bandits. - Gaussian process bandit regret (26 papers)
Tighten regret bounds for GP-UCB and Thompson sampling in Gaussian process optimization.
Most cited and most cited since 2024:
- Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations (NeurIPS 2016 · 63 citations)
- Improved Regret Bounds for Thompson Sampling in Linear Quadratic Control Problems (ICML 2018 · 52 citations)
- Learning Encodings for Constructive Neural Combinatorial Optimization Needs to Regret (AAAI 2024 · 6 citations)
- The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models (AAAI 2024 · 2 citations)
Related topics in Bandits and online learning
- Online convex optimization and games (1,006)
- Multi-armed bandits (537)
- RL theory and MDPs (275)
