AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Trust Region Evolution Strategies

Guoqing Liu, Li Zhao, Feidiao Yang, Jiang Bian, Tao Qin, Nenghai Yu, Tie-Yan Liu

AAAI 2019 · Reinforcement learning › Policy gradient and actor-critic · approach: Alternative policy update rules · 30 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add data reuse via clipped surrogate objective with multiple epochs (TRES)ES · Swimmer-v1timesteps to threshold: 4.5e+5 -> 2.4e+5Improved
add clipped surrogate objective with multi-epoch reuse (TRES)ES · HalfCheetah-v1timesteps to threshold: 1.7e+6 -> 9.8e+5Improved
add clipped surrogate objective with multi-epoch reuse (TRES)ES · Walker2d-v1timesteps to threshold: 1.2e+7 -> 3.6e+6Improved
add clipped surrogate objective with multi-epoch reuse (TRES)ES · Ant-v1timesteps to threshold: 1.5e+7 -> 1.3e+7Improved
add clipped surrogate objective with multi-epoch reuse (TRES)ES · InvertedDoublePendulum-v1timesteps to threshold: 8.8e+5 -> 6e+5Improved
TRES vs PPOPPO · Walker2d-v1timesteps to threshold: 1.9e+6 -> 3.6e+6Hurt
vary clip factor lambda away from 14.0 (too small or too large) (ablation)TRES · HalfCheetah-v1training curve returnHurt
clip factor lambda in [8.0, 20.0] vs baseline ES (ablation)TRES · HalfCheetah-v1sample efficiencyImproved
increase epoch number K to large values (ablation)TRES · HalfCheetah-v1training curve returnHurt
use K=15 epochs rather than small K (ablation)TRES · HalfCheetah-v1training curve returnImproved
multi-epoch updates in TRES vs single update in ESTRES · HalfCheetah-v1wall-clock training timeNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy