Research map / Reinforcement learning / Policy gradient and actor-critic
IPO: Interior-Point Policy Optimization under Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IPO instead of CPO | constrained policy optimization · Point Gather, Point Circle, HalfCheetah-Safe; discounted cumulative constraints | reward and constraint cost | Helped |
| Use IPO instead of PDO | constrained policy optimization · Point Gather, Point Circle, Mars Rover; mean-valued constraints | constraint satisfaction | Helped |
| Use IPO instead of PDO | constrained policy optimization · Point Gather, Point Circle, Mars Rover; mean-valued constraints | training variance | Helped |
| Use unconstrained PPO or TRPO instead of a constraint-aware method | policy optimization · Point Gather, Point Circle, HalfCheetah-Safe | constraint cost | Hurt |
| Use IPO instead of CPO with a loose constraint | Point Gather · Point Gather, constraint threshold 1 | average bombs collected: around $1$ -> around $0.25$ | Helped |
| Increase the barrier parameter t (ablation) | IPO · Point Gather | reward | Helped |
| Increase the barrier parameter t (ablation) | IPO · Point Gather | constraint cost | Hurt |
| Increase the barrier parameter t (ablation) | IPO · constrained RL tasks | convergence rate | Hurt |
| Change PDO's initial Lagrange multiplier from 0.01 to 0.1 (ablation) | PDO · Point Gather | performance | Mixed |
| Change PDO's learning rate from 0.01 to 0.001 (ablation) | PDO · Point Gather | convergence pace | Hurt |
| Add action-outcome noise with scale factor 0.5 (ablation) | IPO · stochastic environment | constraint satisfaction | No clear effect |
| Add a second cumulative constraint and its barrier term (ablation) | IPO · Point Gather, three two-constraint settings | constraint satisfaction | Helped |
Closest papers by meaning
- ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints (ICML 2024 · similarity 0.87)
- Reward Constrained Policy Optimization (ICLR 2019 · similarity 0.87)
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- P2BPO: Permeable Penalty Barrier-Based Policy Optimization for Safe RL (AAAI 2024 · similarity 0.85)
- Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively (AAAI 2024 · similarity 0.85)
- DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.