Research map / Reinforcement learning / Policy gradient and actor-critic
Differentiable Meta-Learning of Bandit Policies
Craig Boutilier, Chih-wei Hsu, Branislav Kveton, Martin Mladenov, Csaba Szepesvari, Manzil Zaheer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use baseline subtraction b_SELF or b_OPT instead of no baseline in the policy gradient (ablation) | GradBand · 2-armed Bernoulli bandit, Exp3 and SoftElim | gradient variance | Improved |
| use b_SELF baseline instead of no baseline when optimizing Exp3 (ablation) | Exp3 with GradBand · 2-armed Bernoulli bandit | Bayes regret / iterations to converge | Improved |
| use b_OPT vs b_SELF baseline for SoftElim (ablation) | SoftElim with GradBand · 2-armed Bernoulli bandit | Bayes regret | No clear effect |
| use SoftElim instead of Exp3 as the parameterized policy | GradBand · 2-armed Bernoulli bandit | Bayes regret: 10.96 -> 4.74 | Improved |
| tune SoftElim by GradBand instead of using Thompson sampling | SoftElim · 2-armed Bernoulli bandit | Bayes regret: 5.47 -> 4.74 | Improved |
| learn Exp3 exploration w by GradBand versus UCB1 | Exp3 · 2-armed Bernoulli bandit | Bayes regret: 9.95 -> 10.96 | Hurt |
| use RNN policy trained with curriculum (n=20 then 200) instead of Thompson sampling and Gittins index | RNN policy · 2 arms with Delta=0.2 prior over two instances | Bayes regret | Improved |
| use RNN policy instead of TS under Beta(1,1) prior where Gittins is optimal | RNN policy · 2 arms Beta(1,1) prior | Bayes regret | Improved |
| increase number of distractor arms K for RNN policy | RNN policy · better-arm bias bandit | Bayes regret | No clear effect |
| use DQN instead of GradBand-optimized policies | DQN · 2-armed Bernoulli bandit | Bayes regret: 4.74 -> 16.81 | Hurt |
Closest papers by meaning
- Meta-Learning for Simple Regret Minimization (AAAI 2023 · similarity 0.92)
- Meta-learning with Stochastic Linear Bandits (ICML 2020 · similarity 0.91)
- Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates (NeurIPS 2024 · similarity 0.89)
- A Classification View on Meta Learning Bandits (ICML 2025 · similarity 0.89)
- Causal Bandits: The Pareto Optimal Frontier of Adaptivity, a Reduction to Linear Bandits, and Limitations around Unknown Marginals (ICML 2024 · similarity 0.89)
- Forced Exploration in Bandit Problems (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
