AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions

Antonio Terpin, Nicolas Lanzetti, Batuhan Yardim, Florian Dorfler, Giorgia Ramponi

NeurIPS 2022 · Reinforcement learning › Policy gradient and actor-critic · approach: Tree search and policy optimization · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace GAE with a neural network advantage approximation (ablation)OT-TRPO · Hopper-v3trained agent score: 2939 -> 327Hurt
increase number of sampled actions for NN advantage from 2 to 32 (ablation)OT-TRPO with NN advantage · Hopper-v3trained agent score: 327 -> 497No clear effect
larger trust region radius epsilon (ablation)OT-TRPO · Taxi-v3learning curveMixed
smaller trust region radius epsilon (ablation)OT-TRPO · Taxi-v3learning speedHurt
very cheap or very expensive Passenger transport cost (ablation)OT-TRPO · Taxi-v3learning performanceHurt
cheap (-0.2) vs equally costly transport cost (ablation)OT-TRPO · Taxi-v3learning speedImproved
expensive (+0.2) vs equally costly transport cost (ablation)OT-TRPO · Taxi-v3learning speedHurt
GAE lambda away from 1 (biased advantage) (ablation)OT-TRPO · continuous controltraining stabilityHurt
mass splitting (OT-TRPO) vs transport-map update (WPO)OT-TRPO · Taxi-v3trust region violation: 72Improved
OT-TRPO vs best alternativeOT-TRPO · Swimmer-v3trained agent performance: 50Improved
OT-TRPO vs BGPG and WNPGOT-TRPO · continuous controlfinal returnImproved
OT-TRPO vs TRPO under KL when policies differ in supportTRPO · CliffWalking-v0convergenceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy