Research map / Reinforcement learning / Policy gradient and actor-critic
Consistent On-Line Off-Policy Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use COP-TD instead of ETD | On-line off-policy evaluation · Chain MDP, random MDP, acrobot, and cart-pole | value-function estimation error | Mixed |
| Use COP-TD or Log-COP-TD instead of GTD | On-line off-policy evaluation · Chain MDP, random MDP, acrobot, and cart-pole | value-function estimation error | Helped |
| Use COP-TD or Log-COP-TD instead of ETD | On-line off-policy evaluation · Chain MDP with large behavior-target stationary-distribution discrepancy | value-function estimation error | Helped |
| Add linear position features when estimating the state-distribution ratio (ablation) | Log-COP-TD · Chain MDP | ρ_d estimation accuracy | Helped |
| Use Log-COP-TD instead of COP-TD for the state-distribution ratio | State-distribution ratio estimation · Mountain car, rarely observed states | ρ_d estimation error | Mixed |
| Increase β (ablation) | COP-TD · Random MDP with 32 states | ρ_d estimation SSE | Hurt |
| Increase or decrease γ_log too far from a suitable value (ablation) | Log-COP-TD · Chain MDP and random MDP | ρ_d estimation SSE | Hurt |
| Tune γ_log (ablation) | Log-COP-TD · Chain MDP and random MDP | ρ_d estimation SSE | Helped |
Closest papers by meaning
- Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples (NeurIPS 2019 · similarity 0.90)
- The Pitfalls of Regularization in Off-Policy TD Learning (NeurIPS 2022 · similarity 0.89)
- A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning (NeurIPS 2025 · similarity 0.88)
- Revisiting a Design Choice in Gradient Temporal Difference Learning (ICLR 2025 · similarity 0.88)
- Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness (ICML 2024 · similarity 0.87)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.