Research map / Optimization / Optimizers
Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the unbiased SF estimate with LSF | N-sample meta-RL gradient estimation · N-sample meta-RL objective | gradient-estimate variance: Θ(N) -> 𝒪(1/N) | Helped |
| Use LSF | N-sample meta-RL gradient estimation · N-sample meta-RL objective | gradient bias: 𝒪(1/√N) | Hurt |
| Use LSF instead of SF | 1-D optimization problem · 1-D optimization problem, large N | final objective after fixed ascent updates | Helped |
| Use LSF instead of SF | 1-D optimization problem · 1-D optimization problem, small N | final objective after fixed ascent updates | Hurt |
| Use LSF instead of SF | Meta-RL gradient estimation · High-dimensional meta-RL tasks tested | learning performance | Helped |
| Increase N while using SF | 1-D optimization problem · 1-D optimization problem | mean-square error | Hurt |
| Increase N while using LSF | 1-D optimization problem · 1-D optimization problem | mean-square error | Helped |
| Use PW when reparameterization is available instead of LSF | 1-D optimization problem · 1-D optimization problem | mean-square error | Helped |
| Use a biased estimate of the value-function Hessian | Meta-RL gradient estimation · Meta-RL | performance | Helped |
Closest papers by meaning
- A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning (NeurIPS 2022 · similarity 0.94)
- Distributional Meta-Gradient Reinforcement Learning (ICLR 2023 · similarity 0.91)
- Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation (NeurIPS 2021 · similarity 0.91)
- Taming MAML: Efficient unbiased meta-reinforcement learning (ICML 2019 · similarity 0.90)
- Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator (NeurIPS 2024 · similarity 0.88)
- Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.