AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

First Order Constrained Optimization in Policy Space

Yiming Zhang, Quan Vuong, Keith Ross

NeurIPS 2020 · Reinforcement learning › Policy gradient and actor-critic · approach: Constrained policy optimization

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use FOCOPS instead of CPOsafe RL on speed-limited robots · Ant-v3reward return: 1406.0 -> 1830.0Improved
use FOCOPS instead of CPOsafe RL on speed-limited robots · Humanoid-v3reward return: 3952.7 -> 4529.7Improved
use FOCOPS instead of PPO-Lsafe RL on speed-limited robots · Humanoid-v3reward return: 471.3 -> 4529.7Improved
use FOCOPS instead of TRPO-Lsafe RL on speed-limited robots · HalfCheetah-v3reward return: 1621.59 -> 1612.2No clear effect
use FOCOPS instead of TRPO-Lsafe RL on speed-limited robots · Hopper-v3reward return: 750.3 -> 1953.4Improved
use FOCOPS instead of TRPO-Lsafe RL on speed-limited robots · HalfCheetah-v3cost return: 164.93 -> 152.36Improved
use FOCOPS instead of PPO-Lcircle tasks · Ant-Circlereward return: 637.4 -> 965.9Improved
use FOCOPS instead of CPOcircle tasks · Humanoid-Circlereward return: 671.0 -> 1106.1Improved
evaluate trained agent on unseen seedsFOCOPS vs other constrained algorithms · Hopper-v3 generalizationreward return: 2749.9 vs 2446.2Hurt
evaluate trained agent on unseen seedsFOCOPS vs other constrained algorithms · Ant-v3 generalizationreward return: 1721.4 -> 1934.9Improved
set nu_max to +infinity instead of 2 (ablation)FOCOPS · speed-limit tasksperformance: 0.3% degradationNo clear effect
use lambda other than 1.5 (ablation)FOCOPS · speed-limit tasksperformance: 7.4% worseHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy