Research map / Reinforcement learning / Policy gradient and actor-critic
Constrained Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CPO instead of primal-dual optimization | constrained policy search · simulated robot locomotion tasks | constraint satisfaction during training | Helped |
| Add cost shaping that penalizes predicted entry into unsafe states (ablation) | CPO · Ant-Circle and Ant-Gather | true constraint return | Helped |
| Initialize the primal-dual constraint variable to 1000 instead of 0 (ablation) | PDO · Point environments | reward return | Hurt |
| Use fixed penalty coefficients instead of CPO's per-update constraint optimization (ablation) | TRPO with shaped constraint costs · Ant-Circle | reward and constraint trade-off | Mixed |
| Set the constraint-advantage GAE parameter to 1 in higher-dimensional environments rather than below 1 (ablation) | CPO · higher-dimensional environments | constraint satisfaction | Hurt |
| Use a constraint-advantage GAE parameter of 0.92 rather than 0.5 (ablation) | CPO · higher-dimensional environments | constraint satisfaction | No clear effect |
| Use constraints instead of an unconstrained reward objective | CPO · simulated robot locomotion tasks | constraint satisfaction | Helped |
Closest papers by meaning
- Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction (NeurIPS 2026 · similarity 0.92)
- Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.90)
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning (AAAI 2024 · similarity 0.88)
- Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively (AAAI 2024 · similarity 0.88)
- Going Beyond Heuristics by Imposing Policy Improvement as a Constraint (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.