Research map / Reinforcement learning / Policy gradient and actor-critic
Trajectory-Based Off-Policy Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| incorporate previously collected deterministic-policy rollouts through trajectory replay (ablation) | DD-OPG · Cartpole | learning speed | Helped |
| fully optimize the surrogate model instead of taking one gradient step (ablation) | DD-OPG · Cartpole | learning speed | Helped |
| use longer surrogate lengthscales (ablation) | DD-OPG · Cartpole | learning speed | Helped |
| set trajectory-selection temperature to an intermediate level rather than too high or too low (ablation) | DD-OPG · Cartpole | learning performance | Helped |
| use prioritized trajectory replay with a mix of previously seen and current trajectories | DD-OPG · Swimmer | learning performance | Mixed |
| reuse off-policy data and optimize the deterministic surrogate | DD-OPG · Cartpole, Mountaincar, Swimmer | learning speed | Helped |
| reuse off-policy data and optimize the deterministic surrogate | DD-OPG · Cartpole, Mountaincar, Swimmer | final performance | No clear effect |
Closest papers by meaning
- Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (NeurIPS 2017 · similarity 0.91)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.91)
- Off-Policy Deep Reinforcement Learning without Exploration (ICML 2019 · similarity 0.91)
- Statistically Efficient Off-Policy Policy Gradients (ICML 2020 · similarity 0.90)
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Identifying Policy Gradient Subspaces (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.