Research map / Reinforcement learning / Policy gradient and actor-critic
Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the black-box backward-flow estimator instead of naive averaging | Off-policy estimation on ModelWin · ModelWin | RMSE | Helped |
| Use the black-box backward-flow estimator instead of IPS | Off-policy estimation on ModelWin with short behavior trajectories · ModelWin | RMSE | Helped |
| Use the black-box estimator instead of the compared baselines | Off-policy estimation on classic control tasks · Pendulum, Mountain Car, Cartpole, Acrobot | RMSE | Mixed |
| Use the black-box estimator instead of IPS on short trajectories | Off-policy estimation on ModelWin · ModelWin, trajectory length 4 | bias | Helped |
| Use the black-box estimator instead of the compared methods as the behavior policy diverges from the target | Off-policy estimation on classic control tasks · Classic control, varying behavior-policy mixture | RMSE | Helped |
| Lower DualDICE's discount factor | DualDICE on ModelWin · ModelWin | bias | Hurt |
| Lower DualDICE's discount factor | DualDICE on ModelWin · ModelWin | variance | Helped |
| Use a behavior policy close to random, leaving Mountain Car unable to reach the goal | All compared off-policy estimators · Mountain Car, alpha_1=0.1 | RMSE | No clear effect |
Closest papers by meaning
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation (NeurIPS 2018 · similarity 0.93)
- Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (NeurIPS 2019 · similarity 0.93)
- Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies (NeurIPS 2020 · similarity 0.91)
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.89)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Dynamic Uncertainty Estimation for Offline Reinforcement Learning (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.