Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Online Posterior Sampling with a Diffusion Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaplaceDPS and DiffTS, approximate posterior sampling for linear bandits and GLMs using a diffusion model prior, where each reverse-diffusion stage's conditional posterior is obtained in closed form or via Laplace approximation.
Problem
Thompson sampling in contextual bandits is limited to Gaussian priors, which cannot represent complex or multimodal distributions over model parameters; existing diffusion posterior sampling methods based on likelihood scores become unstable as observations grow.
Approach
Sample from a chain of approximate conditional posteriors, one per reverse diffusion stage, where each conditional is a product of the pretrained conditional prior and a Gaussian evidence term (closed form for linear models, Laplace/IRLS for GLMs), so no likelihood gradient is needed and the method is asymptotically consistent.
Key results
- DiffTS achieves lower regret than TS, TunedTS, MixTS and DPS on synthetic and MovieLens bandit problems, while DPS diverges as rounds increase.
- DiffTS posterior approximations have earth mover's distance comparable to MixTS, which has the exact posterior in the Gaussian mixture setting.
- DiffTS costs about 12 seconds per 100 runs versus 0.1 seconds for Gaussian TS, with computation linear in the number of diffusion stages T.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the diffusion model prior with DiffTS instead of the reported baselines | contextual bandits on the synthetic problems · Synthetic problems | regret | Helped |
| use the diffusion model prior with DiffTS instead of the reported baselines | linear and logistic contextual bandits · MovieLens | regret | Helped |
| use the diffusion model prior with DiffTS instead of the reported baselines | linear and logistic contextual bandits · MNIST | regret | Helped |
| use likelihood-score guidance in DPS | DPS · Synthetic contextual bandits | regret | Hurt |
| increase diffusion-prior training samples (ablation) | DiffTS · Synthetic cross | regret | Helped |
| increase the number of diffusion stages from T=1 to T=50 (ablation) | DiffTS · Synthetic cross | regret: 26 at T = 1 to 15 at T = 50 | Helped |
| increase diffusion stages T (ablation) | DiffTS · Synthetic cross | computation time | Hurt |
| use a learned Gaussian mixture prior instead of a learned Gaussian prior | TunedTS and MixTS · MovieLens | regret | No clear effect |
Surprising findings
- The DPS likelihood-score baseline diverges as the number of rounds grows, even after tuning its step size.
- MixTS barely outperforms TunedTS on the higher-dimensional MovieLens problem.
Limitations
- Computational cost scales roughly linearly with the number of diffusion stages T.
- Performance depends on the number of training samples for the diffusion prior and on T; with few samples it performs like Gaussian TS.
- No regret bound is proved; only asymptotic consistency of the posterior approximation is shown.
Experimental setup
Datasets: MovieLens 1M MNIST
Benchmarks: synthetic 2D linear bandit problems MovieLens linear and logistic bandits MNIST linear and logistic bandits Gaussian mixture non-bandit posterior evaluation
Metrics: cumulative regret earth mover's distance to true posterior compute time
Baselines: TS TunedTS MixTS DPS SMC
Builds on: Thompson sampling diffusion posterior sampling (DPS) Denoising diffusion probabilistic models iteratively reweighted least squares linear bandits
Closest papers by meaning
- Diffusion Models Meet Contextual Bandits (NeurIPS 2025 · similarity 0.92)
- Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown (NeurIPS 2025 · similarity 0.88)
- $\mathtt{VITS}$ : Variational Inference Thompson Sampling for contextual bandits (ICML 2024 · similarity 0.88)
- The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models (AAAI 2024 · similarity 0.87)
- Active Context Selection Improves Simple Regret in Contextual Bandits (NeurIPS 2026 · similarity 0.87)
- Local Anti-Concentration Class: Logarithmic Regret for Greedy Linear Contextual Bandit (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.