Research map / Reinforcement learning / Policy gradient and actor-critic
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use doubly robust (DR) estimator instead of step-wise importance sampling (IS) | off-policy value evaluation (Mountain Car, point estimation) · Mountain Car | estimation error (MSE) | Helped |
| use DR with a state-action independent constant Q-hat (DR-bsl) instead of IS | off-policy value evaluation (Mountain Car) · Mountain Car | estimation error (MSE) | Helped |
| use DR-v2 (transition-model correction, P-hat=P assumption, 2-fold) instead of IS/WIS/regression | off-policy value evaluation (KDD Cup 1998 donation simulator) · KDD Cup 1998 donation | estimation error | Helped |
| use DR with 2-fold cross-fitting (k-fold DR) instead of single-split DR (ablation) | off-policy value evaluation (Sailing) · Sailing | estimation error (MSE) | Helped |
| use DR as drop-in replacement for IS in safe policy improvement (lower confidence bound selection) | safe policy improvement (Mountain Car, pi_train maximizing value) · Mountain Car safe policy improvement | true value improvement over behavior policy | Helped |
| use DR instead of IS in safe policy improvement when pi_train is poor (minimizing value) | safe policy improvement (Mountain Car, pi_train minimizing value) · Mountain Car safe policy improvement | safety of recommended policy | No clear effect |
| use DR-v2 with P-hat=P approximation (variance reduction for state transitions) vs original DR on Sailing | DR-v2 on Sailing · Sailing | estimation error | No clear effect |
Closest papers by meaning
- More Robust Doubly Robust Off-policy Evaluation (ICML 2018 · similarity 0.95)
- Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning (NeurIPS 2019 · similarity 0.93)
- More Efficient Off-Policy Evaluation through Regularized Targeted Learning (ICML 2019 · similarity 0.93)
- Doubly Optimal Policy Evaluation for Reinforcement Learning (ICLR 2025 · similarity 0.90)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.90)
- Dynamic Uncertainty Estimation for Offline Reinforcement Learning (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.