Research map / Reinforcement learning / Policy gradient and actor-critic
Phasic Policy Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPG’s alternating policy and auxiliary phases with value-feature distillation instead of PPO | PPO on Procgen · Procgen Benchmark | sample efficiency | Helped |
| Use a single policy epoch rather than additional policy epochs (ablation) | PPG · Procgen Benchmark | performance | No clear effect |
| Increase auxiliary-phase training to about six epochs (ablation) | PPG · Procgen Benchmark | performance | Helped |
| Perform auxiliary phases too frequently (ablation) | PPG · Procgen Benchmark | performance | Hurt |
| Replace PPO clipping with a fixed KL-penalty policy objective (ablation) | PPG · Procgen Benchmark | performance | No clear effect |
| Use one shared network with gradient detachment instead of separate policy and value networks (ablation) | PPG · Procgen Benchmark | performance | No clear effect |
| Move auxiliary-phase value-function optimization into the policy phase (ablation) | PPG · Procgen Benchmark | performance | No clear effect |
| Add extra value-function optimization during PPO policy updates by increasing sample reuse (ablation) | PPO · Procgen Benchmark | performance | Helped |
Closest papers by meaning
- PPG Reloaded: An Empirical Study on What Matters in Phasic Policy Gradient (ICML 2023 · similarity 0.93)
- Towards Convergence of PPO: An Approximate Descent Approach (NeurIPS 2026 · similarity 0.88)
- Low-Switching Policy Gradient with Exploration via Online Sensitivity Sampling (ICML 2023 · similarity 0.88)
- SAPG: Split and Aggregate Policy Gradients (ICML 2024 · similarity 0.87)
- Convergence Analysis of Policy Gradient Methods with Dynamic Stochasticity (ICML 2025 · similarity 0.87)
- TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.