Research map / Reinforcement learning / Offline RL
Offline RL Without Off-Policy Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| one-step Rev. KL regularization instead of best iterative algorithm from prior work | offline RL · hopper-medium | normalized return: 58.8 -> 83.3 | Helped |
| one-step Rev. KL instead of best iterative | offline RL · halfcheetah-medium-expert | normalized return: 64.7 -> 93.5 | Helped |
| one-step instead of best iterative on random data | offline RL · halfcheetah-random | normalized return: 35.4 -> 6.9 | Hurt |
| one-step instead of multi-step policy improvement (same Rev KL operator) (ablation) | reverse KL regularized offline RL · hopper-medium | normalized return: 53.0 -> 83.3 | Helped |
| one-step instead of iterative actor-critic (same Rev KL operator) (ablation) | reverse KL regularized offline RL · hopper-medium-expert | normalized return: 82.7 -> 102.1 | Helped |
| one-step instead of iterative on halfcheetah-random (ablation) | reverse KL regularized offline RL · halfcheetah-random | normalized return: 16.3 -> 6.9 | Hurt |
| one-step instead of iterative on halfcheetah-medium-expert (ablation) | reverse KL regularized offline RL · halfcheetah-medium-expert | normalized return: 93.6 -> 93.5 | No clear effect |
| Rev. KL vs Easy BCQ vs exponential weighting as improvement operator (ablation) | one-step algorithm · D4RL gym-mujoco | normalized return | Mixed |
| one-step less sensitive to regularization strength than iterative or multi-step (ablation) | one-step algorithm · halfcheetah-medium | robustness to alpha | Helped |
Closest papers by meaning
- A Minimalist Approach to Offline Reinforcement Learning (NeurIPS 2021 · similarity 0.94)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
- Behavior Proximal Policy Optimization (ICLR 2023 · similarity 0.92)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.92)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.