Research map / Reinforcement learning / Policy gradient and actor-critic
Learning to Constrain Policy Optimization with Virtual Trust Region
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a learned virtual-policy trust region in addition to the old-policy region (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: 100 or 50 return | Helped |
| Replace dynamic trust-region weighting with fixed α=0.5 (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: 100 or 50 return | Hurt |
| Use only the virtual-policy trust region (α=1.0) (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: 100 or 50 return | Hurt |
| Replace switching-β with annealed β (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: about 50 return | Hurt |
| Replace switching-β with PPO-style adaptive β (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: 200 return | Hurt |
| Write policies to memory at every update (ablation) | MCPO · BipedalWalkerHardcore-v3 | return | Hurt |
| Use fixed-interval memory writing every 10 or 100 updates (ablation) | MCPO · BipedalWalkerHardcore-v3 | return | Hurt |
| Replace learned virtual policy with Mean ψ (ablation) | MCPO · BipedalWalkerHardcore-v3 | return | Hurt |
| Generate the virtual policy from only half the context features (ablation) | MCPO · BipedalWalkerHardcore-v3 | return: 50 return | Hurt |
| Increase memory size from N=5 to N=40 (ablation) | MCPO · Pendulum, LunarLander, BipedalWalker | return: -133.42\textpm 4.53; -135.57\textpm 5.22; 267.19\textpm 13.42; 249.51\textpm 12.75 | Mixed |
| Replace attention-selected virtual policy with the mean of past policies (ablation) | MCPO (N=40) · Humanoid and HumanoidStandup | return: 353\textpm 27; 71,308\textpm 11,113 | Hurt |
Closest papers by meaning
- Embedding Safety into RL: A New Take on Trust Region Methods (ICML 2025 · similarity 0.89)
- Going Beyond Heuristics by Imposing Policy Improvement as a Constraint (NeurIPS 2024 · similarity 0.88)
- Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints (ICML 2024 · similarity 0.88)
- Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction (NeurIPS 2026 · similarity 0.88)
- Reflective Policy Optimization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.