Research map / Reinforcement learning / Policy gradient and actor-critic
Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the second-order TayPO estimate instead of the first-order estimate | Hessian estimation in tabular MDPs · Random tabular MDPs | gradient/Hessian estimate accuracy | Helped |
| Use the first-order estimate instead of the second-order estimate (ablation) | Hessian estimation in tabular MDPs · Random tabular MDPs with small sample size | Hessian estimate accuracy | Helped |
| Increase the sample size for the first-order estimate | Hessian estimation in tabular MDPs · Random tabular MDPs | Hessian estimate accuracy | Mixed |
| Use the doubly-robust estimate instead of step-wise importance sampling | Higher-order derivative estimation · Random tabular MDPs | estimate accuracy | Helped |
| Truncate importance ratios in the DR estimate (ablation) | Higher-order derivative estimation · Random tabular MDPs | estimate accuracy | Hurt |
| Use the second-order estimate rather than the first-order estimate (ablation) | PROMP · 2-D navigation | training stability | Helped |
| Use the second-order TayPO estimate rather than the default first-order estimate (ablation) | PROMP · MuJoCo locomotion tasks | learning speed and final performance | Mixed |
| Use the first-order MAML approximation rather than carrying out Hessian approximations (ablation) | TRPO-FMAML · MuJoCo locomotion tasks | training performance | Hurt |
| Use TRPO-DiCE rather than PROMP | Meta-RL training · MuJoCo locomotion tasks | training performance | Hurt |
Closest papers by meaning
- Statistically Efficient Off-Policy Policy Gradients (ICML 2020 · similarity 0.92)
- Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learning (ICML 2022 · similarity 0.91)
- A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning (NeurIPS 2022 · similarity 0.91)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.89)
- Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator (NeurIPS 2024 · similarity 0.89)
- A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.