Research map / Reinforcement learning / Offline RL
REG: In-Sample RL via Regularizing the Evaluation Gap
Hanpu Shen, Weining Shen, Roy Fox
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use orthogonal policy gradient (OPG) instead of weighted behavior cloning only for policy extraction (ablation) | REG · D4RL MuJoCo locomotion (medium, medium-replay, medium-expert) | average normalized score | Improved |
| use OPG instead of WBC-only on medium-expert datasets (ablation) | REG · D4RL MuJoCo medium-expert | average normalized score | Improved |
| increase value conservatism alpha on medium and medium-replay datasets (ablation) | REG · D4RL MuJoCo medium and medium-replay | normalized score | Improved |
| lower alpha on expert datasets (ablation) | REG · D4RL MuJoCo medium-expert | normalized score | No clear effect |
| set OPG strength lambda to an intermediate value in [0.2, 0.5] rather than lower or higher (ablation) | REG · D4RL MuJoCo medium and medium-replay | normalized score: peak at lambda in [0.2, 0.5] | Improved |
| use excessively large lambda for the orthogonal gradient (ablation) | REG · D4RL expert-level datasets | normalized score | Hurt |
| use Gaussian-policy REG instead of Gaussian-policy IQL | IQL · D4RL AntMaze average | normalized score: 57.2 -> 77.5 | Improved |
| use REG instead of IVR | IVR · D4RL Locomotion average | normalized score: 83.3 -> 87.8 | Improved |
| use REG instead of ReBRAC | ReBRAC · D4RL Locomotion average | normalized score: 88.4 -> 87.8 | No clear effect |
| use REG with Gaussian policy instead of IDQL diffusion policy | IDQL · D4RL AntMaze average | normalized score: 79.2 -> 77.5 | No clear effect |
Closest papers by meaning
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning (ICML 2025 · similarity 0.90)
- Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Relaxed Stationary Distribution Correction Estimation for Improved Offline Policy Optimization (AAAI 2024 · similarity 0.90)
- Efficient Multi-Policy Evaluation for Reinforcement Learning (AAAI 2025 · similarity 0.90)
- Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
