Research map / Reinforcement learning / Policy gradient and actor-critic
Fast Efficient Hyperparameter Tuning for Policy Gradient Methods
Supratik Paul, Vitaly Kurin, Shimon Whiteson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt the learning rate online with HOOF | A2C · MuJoCo continuous control tasks | learning speed | Improved |
| Adapt the learning rate online with HOOF | A2C · HalfCheetah and Walker | performance | Improved |
| Use HOOF instead of meta-gradients with a grid-tuned meta learner | A2C · MuJoCo continuous control tasks | performance: 36 times as many samples | Improved |
| Vary the KL constraint epsilon across the tested range (ablation) | HOOF with A2C · HalfCheetah, Hopper, Ant, and Walker | return | No clear effect |
| Remove the KL constraint (ablation) | HOOF with A2C · MuJoCo continuous control tasks | convergence | Hurt |
| Use a KL constraint to keep candidate policy updates bounded | First-order HOOF · MuJoCo continuous control tasks | WIS estimate reliability | Improved |
| Use a (gamma, lambda)-conditioned value function | HOOF-TNPG · HalfCheetah, Hopper, and Walker | performance | Improved |
| Remove (gamma, lambda) conditioning from the value function (ablation) | HOOF-TNPG · HalfCheetah, Hopper, and Walker | performance | Hurt |
| Remove (gamma, lambda) conditioning from the value function (ablation) | HOOF-TNPG · Ant | performance | No clear effect |
| Learn delta, gamma, and lambda with HOOF | TNPG · HalfCheetah, Hopper, and Ant | learning speed | Improved |
| Learn delta, gamma, and lambda with HOOF instead of using default TRPO settings | TNPG · HalfCheetah, Hopper, and Ant | performance | Improved |
| Learn delta, gamma, and lambda with HOOF instead of using default TRPO settings | TNPG · Walker | performance | No clear effect |
Closest papers by meaning
- Bayesian Optimization for Iterative Learning (NeurIPS 2020 · similarity 0.87)
- Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL (NeurIPS 2021 · similarity 0.87)
- A Parametric Class of Approximate Gradient Updates for Policy Optimization (ICML 2022 · similarity 0.87)
- Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning (NeurIPS 2025 · similarity 0.87)
- Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies (ICML 2023 · similarity 0.87)
- A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
