Research map / Bandits and online learning / Contextual bandits and Thompson sampling
The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how noninformative priors affect Thompson sampling's asymptotic regret in uniform bandits with unknown support, shows uniform-prior optimality depends on parameterization, and proposes TS with Truncation (TS-T) that is asymptotically optimal for uniform and Gaussian bandits under reference and Jeffreys priors.
Problem
Thompson sampling's asymptotic optimality depends on the choice of noninformative prior in multiparameter bandit models, and standard priors (uniform, Jeffreys, reference) are either parameterization-dependent or suboptimal.
Approach
Extends TS regret analysis to uniform bandits with unknown support, shows uniform priors are optimal only under specific parameterizations, and proposes TS-T, which lower-bounds the scale estimate (max(x_1 + 1/n, x_n) - x_1, or max(1, S) for Gaussian) so invariant priors (reference, Jeffreys) achieve the asymptotic lower bound.
Key results
- Under the uniform bandit with location-scale parameterization, TS with the uniform prior is asymptotically optimal, while TS with Jeffreys and reference priors is suboptimal (k>=1 incurs at least logarithmic regret in expectation for small sigma_2).
- TS-T with reference or Jeffreys priors is proven asymptotically optimal for uniform and Gaussian bandits via finite-time regret bounds.
- In the 6-armed uniform simulation over 10,000 runs, TS with the uniform prior performs best while TS with reference prior shows large regret, and TS-T with the reference prior improves significantly.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the uniform prior with location-scale parameterization instead of the reference or Jeffreys prior | TS on uniform bandits · Uniform bandits, location-scale parameterization | asymptotic regret optimality | Helped |
| Use the uniform prior with location-rate parameterization instead of location-scale parameterization | TS on uniform bandits · Uniform bandits | asymptotic regret optimality | Hurt |
| Use the Jeffreys prior instead of the uniform prior with location-scale parameterization | TS on uniform bandits · Uniform bandits, location-scale parameterization | asymptotic regret optimality | Hurt |
| Use the reference prior instead of the uniform prior with location-scale parameterization | TS on uniform bandits · Uniform bandits, location-scale parameterization | asymptotic regret optimality | Hurt |
| Use the uniform prior with location-scale parameterization instead of the Jeffreys or reference prior | TS on Gaussian bandits · Gaussian bandits, location-scale parameterization | asymptotic regret optimality | Helped |
| Use the Jeffreys prior instead of the uniform prior with location-scale parameterization | TS on Gaussian bandits · Gaussian bandits, location-scale parameterization | asymptotic regret optimality | Hurt |
| Use the reference prior instead of the uniform prior with location-scale parameterization | TS on Gaussian bandits · Gaussian bandits, location-scale parameterization | asymptotic regret optimality | Hurt |
| Use the uniform prior with location-rate parameterization instead of location-scale parameterization | TS on Gaussian bandits · Gaussian bandits | asymptotic regret optimality | Hurt |
| Add adaptive scale truncation (ablation) | TS with Jeffreys or reference prior on uniform bandits · Uniform bandits | asymptotic regret optimality | Helped |
| Add adaptive scale truncation (ablation) | TS with Jeffreys or reference prior on Gaussian bandits · Gaussian bandits | asymptotic regret optimality | Helped |
| Add adaptive scale truncation (ablation) | TS with the reference prior · Six-armed uniform bandit simulation | averaged cumulative regret | Helped |
| Use the uniform prior instead of the Jeffreys or reference prior | TS in the reported uniform bandit simulation · Six-armed uniform bandit simulation | averaged cumulative regret | Helped |
Surprising findings
- Uniform priors under different parameterizations (e.g., location-rate, which equals the Jeffreys prior) can be suboptimal, so the optimality of the uniform prior depends on the parameterization chosen.
Limitations
- Regret upper bound depends on the unknown scale sigma, complicating worst-case (minimax) analysis.
- Theoretical results are restricted to uniform and Gaussian models; the Pareto case with invariant priors remains open.
- TS-T is not strictly a Bayesian policy since truncation makes the sampling distribution not a posterior.
Experimental setup
Benchmarks: 6-armed uniform bandit simulation instance
Metrics: expected cumulative regret asymptotic regret lower bound ratio
Baselines: Thompson sampling with uniform prior Thompson sampling with Jeffreys prior Thompson sampling with reference prior CK-UCB
Builds on: Thompson sampling Lai-Robbins lower bound CK-UCB reference priors Jeffreys prior
Closest papers by meaning
- Optimality of Thompson Sampling with Noninformative Priors for Pareto Bandits (ICML 2023 · similarity 0.95)
- Only Pay for What Is Uncertain: Variance-Adaptive Thompson Sampling (ICLR 2024 · similarity 0.92)
- Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown (NeurIPS 2025 · similarity 0.92)
- Thompson Sampling with Less Exploration is Fast and Optimal (ICML 2023 · similarity 0.92)
- Variance-Aware Feel-Good Thompson Sampling for Contextual Bandits (NeurIPS 2025 · similarity 0.90)
- Thompson Sampling For Combinatorial Bandits: Polynomial Regret and Mismatched Sampling Paradox (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.