Research map / Reinforcement learning / Policy gradient and actor-critic
Off-Policy Deep Reinforcement Learning without Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Constrain candidate actions to actions generated from the batch, with limited perturbation | BCQ · MuJoCo fixed-batch tasks | return | Helped |
| Constrain policy actions to the batch distribution | Off-policy deep RL · Hopper-v1 final buffer, concurrent, and imitation | return | Helped |
| Use a wider perturbation range Φ (ablation) | BCQ · Hopper-v1 imitation | return | Hurt |
| Use a wider perturbation range Φ (ablation) | BCQ · Hopper-v1 imitation | value-function stability | Hurt |
| Use ensemble uncertainty to penalize uncertain actions | Off-policy agent · Hopper-v1 imitation | value-function stability | Helped |
| Use ensemble uncertainty to constrain actions | Off-policy agent · Hopper-v1 imitation | return | Hurt |
| Use a size-10 rather than size-4 value ensemble | Ensemble uncertainty method · Hopper-v1 imitation | uncertainty estimation | Helped |
| Learn from random-policy data rather than expert or learned-policy data | BCQ and DDPG · Pendulum random batch | return | No clear effect |
| Learn from random-policy data | BCQ rather than DDPG · Reacher random batch | return | Helped |
Closest papers by meaning
- Continuous Doubly Constrained Batch Reinforcement Learning (NeurIPS 2021 · similarity 0.92)
- Trajectory-Based Off-Policy Deep Reinforcement Learning (ICML 2019 · similarity 0.91)
- Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (NeurIPS 2017 · similarity 0.90)
- Off-Policy Evaluation via Off-Policy Classification (NeurIPS 2019 · similarity 0.89)
- Policy Regularization with Dataset Constraint for Offline Reinforcement Learning (ICML 2023 · similarity 0.89)
- Efficient Off-Policy Learning for High-Dimensional Action Spaces (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.