AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Differentiable Meta-Learning of Bandit Policies

Craig Boutilier, Chih-wei Hsu, Branislav Kveton, Martin Mladenov, Csaba Szepesvari, Manzil Zaheer

NeurIPS 2020 · Reinforcement learning › Policy gradient and actor-critic · approach: Variance-reduced policy gradients · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use baseline subtraction b_SELF or b_OPT instead of no baseline in the policy gradient (ablation)GradBand · 2-armed Bernoulli bandit, Exp3 and SoftElimgradient varianceImproved
use b_SELF baseline instead of no baseline when optimizing Exp3 (ablation)Exp3 with GradBand · 2-armed Bernoulli banditBayes regret / iterations to convergeImproved
use b_OPT vs b_SELF baseline for SoftElim (ablation)SoftElim with GradBand · 2-armed Bernoulli banditBayes regretNo clear effect
use SoftElim instead of Exp3 as the parameterized policyGradBand · 2-armed Bernoulli banditBayes regret: 10.96 -> 4.74Improved
tune SoftElim by GradBand instead of using Thompson samplingSoftElim · 2-armed Bernoulli banditBayes regret: 5.47 -> 4.74Improved
learn Exp3 exploration w by GradBand versus UCB1Exp3 · 2-armed Bernoulli banditBayes regret: 9.95 -> 10.96Hurt
use RNN policy trained with curriculum (n=20 then 200) instead of Thompson sampling and Gittins indexRNN policy · 2 arms with Delta=0.2 prior over two instancesBayes regretImproved
use RNN policy instead of TS under Beta(1,1) prior where Gittins is optimalRNN policy · 2 arms Beta(1,1) priorBayes regretImproved
increase number of distractor arms K for RNN policyRNN policy · better-arm bias banditBayes regretNo clear effect
use DQN instead of GradBand-optimized policiesDQN · 2-armed Bernoulli banditBayes regret: 4.74 -> 16.81Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy