Research map / Reinforcement learning / Policy gradient and actor-critic
Bayesian Policy Gradient and Actor-Critic Algorithms
Mohammad Ghavamzadeh, Yaakov Engel, Michal Valko
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Bayesian quadrature instead of Monte-Carlo gradient estimate (10 samples) | policy gradient estimation · Bandit r(a)=a | std of gradient estimate: 0.438 -> 0.050 | Improved |
| use Bayesian quadrature instead of Monte-Carlo gradient estimate (100 samples) | policy gradient estimation · Bandit r(a)=a | std of gradient estimate: 0.140 -> 0.000001 | Improved |
| use Bayesian quadrature instead of Monte-Carlo gradient estimate (10 samples) | policy gradient estimation · Bandit r(a)=a^2 | std of gradient estimate: 2.831 -> 0.226 | Improved |
| use Bayesian quadrature instead of Monte-Carlo gradient estimate (100 samples) | policy gradient estimation · Bandit r(a)=a^2 | std of gradient estimate: 0.857 -> 0.000011 | Improved |
| use BQ instead of MC estimate | policy gradient estimation · LQR gradient estimation | MSE of gradient | Improved |
| add Gaussian reward noise (sigma_r = 0.1 or 1) | BQ and MC gradient estimates · LQR gradient estimation | gradient error | Hurt |
| use BPG/BPNG instead of MCPG for sample size M = 5 | policy optimization · LQR | average expected return | Hurt |
| use BPG/BPNG instead of MCPG for sample sizes M = 10, 20, 40 | policy optimization · LQR | average expected return | Improved |
Closest papers by meaning
- Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms (JMLR 2022 · similarity 0.91)
- On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift (JMLR 2021 · similarity 0.90)
- On the Second-Order Convergence of Biased Policy Gradient Algorithms (ICML 2024 · similarity 0.90)
- Augmented Bayesian Policy Search (ICLR 2024 · similarity 0.89)
- Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles (ICML 2024 · similarity 0.89)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
