Research map / Reinforcement learning / Offline RL
Behavior Proximal Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply BPPO rather than BC | BC-initialized policy · D4RL tasks | normalized score: 51% performance improvement | Helped |
| Iteratively improve the already improved behavior policy rather than update it only once (ablation) | BPPO · Gym learning-curve tasks | normalized score | Helped |
| Replace the learned advantage with the behavior-policy advantage (ablation) | BPPO · Gym learning-curve tasks | normalized score | Helped |
| Decay the policy clip ratio during training (ablation) | BPPO · D4RL training tasks | training stability | Helped |
| Use clip ratio ε=0.1 rather than the other tested values (ablation) | BPPO · hopper-medium-replay-v2 | normalized score | Helped |
| Vary clip ratio ε (ablation) | BPPO · hopper-medium-expert-v2 and most environments | normalized score | No clear effect |
| Use ε=0.9 asymmetric advantage weighting rather than equal weighting (ablation) | BPPO · Gym environments | normalized score | Helped |
| Use a low decay rate or no clip-ratio decay (ablation) | BPPO · D4RL training tasks | training stability | Hurt |
Closest papers by meaning
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.91)
- A2PO: Towards Effective Offline Reinforcement Learning from an Advantage-aware Perspective (NeurIPS 2024 · similarity 0.90)
- Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees (ICLR 2024 · similarity 0.90)
- Provable Zero-Shot Generalization in Offline Reinforcement Learning (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.