AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Relative Entropy Pathwise Policy Optimization

Claas A Voelcker, Axel Brunnbauer, Marcel Hussing, Michal Nauman, Pieter Abbeel, Radu Grosu, Eric Eaton, Amir-massoud Farahmand, Igor Gilitschenski

ICLR 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Trajectory reuse in policy gradients · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace pathwise Q gradient with score-function Q gradient (ablation)REPPO · DMC and ManiSkill aggregateaggregate returnHurt
add pathwise gradients through learned Q instead of the PPO GAE clipped objective (ablation)REPPO · DMC and ManiSkill aggregateaggregate returnImproved
add KL regularization of policy updates (ablation)REPPO · DMC design ablationaggregate returnImproved
add HL-Gauss categorical value loss (ablation)REPPO · DMC design ablationaggregate returnImproved
add layer normalization in the critic (ablation)REPPO · DMC design ablationaggregate returnImproved
add auxiliary self-predictive loss (ablation)REPPO · DMC design ablationaggregate returnMixed
shrink the on-policy batch size (ablation)REPPO · DMC batch size studymean aggregate performanceHurt
shrink the replay buffer from default (10,485,760) to 32,768 transitions (ablation)FastTD3 · DMC locomotionaggregate performance: 10,485,760 -> 32,768 transitionsHurt
set GAE lambda to 0.65 instead of 0.95 (ablation)REPPO · Atari-10human-normalized score: 0.95 -> 0.65Improved
use REPPO instead of PPOon-policy RL · DMC and ManiSkill wall-clocknormalized return: about 33% higher normalized returnsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy