Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Leveraging Demonstrations to Improve Online Learning: Quality Matters
Botao Hao, Rahul Jain, Tor Lattimore, Benjamin Van Roy, Zheng Wen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use demonstration action and reward likelihoods with the expert's competence level in the prior | Thompson sampling · Gaussian bandits | cumulative regret | Improved |
| Use offline rewards only and assume the expert chooses actions uniformly at random (ablation) | Thompson sampling · Gaussian bandits | cumulative regret | Improved |
| Increase the expert's deliberateness (ablation) | Informed TS · Gaussian bandits | cumulative regret | Improved |
| Decrease the expert's knowledgeability (ablation) | Informed TS · Gaussian bandits | cumulative regret | Hurt |
| Generate demonstrations with an ε-greedy expert policy instead of the assumed softmax policy (ablation) | Informed TS · Linear bandits with misspecified expert policy | cumulative regret | Improved |
| Use a misspecified expert knowledgeability or deliberateness parameter (ablation) | Informed TS · Linear bandits with misspecified competence | cumulative regret | Mixed |
| Estimate deliberateness from action entropy or maximum likelihood instead of using its true value (ablation) | Informed TS · Linear Gaussian bandit | cumulative regret | Mixed |
| Use maximum-likelihood estimation of deliberateness at large values (ablation) | Informed TS · Linear Gaussian bandit | cumulative regret | Hurt |
| Approximate posterior samples with Bayesian bootstrapping rather than near-exact grid posterior sampling (ablation) | Informed TS · Two-dimensional bandit | cumulative regret | No clear effect |
Closest papers by meaning
- Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity (NeurIPS 2024 · similarity 0.88)
- Bandit Guided Submodular Curriculum for Adaptive Subset Selection (NeurIPS 2025 · similarity 0.86)
- The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models (AAAI 2024 · similarity 0.86)
- Information-Directed Offline-to-Online Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
- Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown (NeurIPS 2025 · similarity 0.86)
- When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
