Research map / Reinforcement learning / Policy gradient and actor-critic
From Importance Sampling to Doubly Robust Policy Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the DR-derived policy-gradient estimator with approximate Q and ∇Q information | Policy-gradient estimation on CartPole · CartPole | variance reduction ratio | Helped |
| Use the DR-derived policy-gradient estimator | Policy optimization on CartPole · CartPole | expected return | Helped |
| Use DR-PG instead of the compared control-variate estimators | Policy optimization on CartPole · CartPole | computational cost | Hurt |
| Add gradient information for the approximate Q function instead of setting ∇Q̃ to zero (ablation) | DR-PG · CartPole | policy-gradient variance | Helped |
| Use approximate Q and ∇Q that accurately match the true values | DR-PG on tree-structured MDPs · Tree-structured MDPs | policy-gradient variance | Helped |
| Use accurate side information in a fully deterministic environment | DR-PG · Deterministic MDPs | policy-gradient variance | Helped |
Closest papers by meaning
- Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling (ICML 2020 · similarity 0.91)
- Minimax Value Interval for Off-Policy Evaluation and Policy Optimization (NeurIPS 2020 · similarity 0.91)
- Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning (NeurIPS 2021 · similarity 0.90)
- State Relevance for Off-Policy Evaluation (ICML 2021 · similarity 0.90)
- Optimal Estimation of Policy Gradient via Double Fitted Iteration (ICML 2022 · similarity 0.90)
- More Robust Doubly Robust Off-policy Evaluation (ICML 2018 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.