Research map / Reinforcement learning / Policy gradient and actor-critic
Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions
Antonio Terpin, Nicolas Lanzetti, Batuhan Yardim, Florian Dorfler, Giorgia Ramponi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GAE with a neural network advantage approximation (ablation) | OT-TRPO · Hopper-v3 | trained agent score: 2939 -> 327 | Hurt |
| increase number of sampled actions for NN advantage from 2 to 32 (ablation) | OT-TRPO with NN advantage · Hopper-v3 | trained agent score: 327 -> 497 | No clear effect |
| larger trust region radius epsilon (ablation) | OT-TRPO · Taxi-v3 | learning curve | Mixed |
| smaller trust region radius epsilon (ablation) | OT-TRPO · Taxi-v3 | learning speed | Hurt |
| very cheap or very expensive Passenger transport cost (ablation) | OT-TRPO · Taxi-v3 | learning performance | Hurt |
| cheap (-0.2) vs equally costly transport cost (ablation) | OT-TRPO · Taxi-v3 | learning speed | Improved |
| expensive (+0.2) vs equally costly transport cost (ablation) | OT-TRPO · Taxi-v3 | learning speed | Hurt |
| GAE lambda away from 1 (biased advantage) (ablation) | OT-TRPO · continuous control | training stability | Hurt |
| mass splitting (OT-TRPO) vs transport-map update (WPO) | OT-TRPO · Taxi-v3 | trust region violation: 72 | Improved |
| OT-TRPO vs best alternative | OT-TRPO · Swimmer-v3 | trained agent performance: 50 | Improved |
| OT-TRPO vs BGPG and WNPG | OT-TRPO · continuous control | final return | Improved |
| OT-TRPO vs TRPO under KL when policies differ in support | TRPO · CliffWalking-v0 | convergence | Hurt |
Closest papers by meaning
- Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference (NeurIPS 2025 · similarity 0.88)
- Deterministic Policy Gradient Primal-Dual Methods for Continuous-Space Constrained MDPs (AAAI 2025 · similarity 0.86)
- Embedding Safety into RL: A New Take on Trust Region Methods (ICML 2025 · similarity 0.86)
- Trust Region Q Adjoint Matching (NeurIPS 2026 · similarity 0.86)
- Relative Policy-Transition Optimization for Fast Policy Transfer (AAAI 2024 · similarity 0.85)
- Wasserstein Policy Optimization (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
