Research map / Reinforcement learning / Policy gradient and actor-critic
Model-Free Trajectory Optimization for Reinforcement Learning
Riad Akrour, Gerhard Neumann, Hany Abdulsamad, Abbas Abdolmaleki
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace linearized dynamics model (GPS) with model-free local quadratic Q-function policy update (MOTO) | trajectory optimization on multi-link swing-up · quadruple link swing-up with restricted joint limits | average return | Improved |
| Replace linearized dynamics model (GPS) with model-free MOTO | trajectory optimization on double link swing-up · double link swing-up (torque limit 25) | average return | Improved |
| Lower entropy reduction constant beta0 from 0.5 to 0.1 (ablation) | MOTO · double link swing-up | average return | Mixed |
| Add importance sampling sample reuse (ablation) | MOTO · double link swing-up (M=20) | convergence speed | Improved |
| Add entropy constraint (ablation) | MOTO · double link swing-up (M=20) | policy quality in last iterations | Mixed |
| Use Monte-Carlo Q estimate instead of dynamic programming estimate (ablation) | MOTO · double link swing-up (M=20) | policy quality | Hurt |
| Reduce rollouts per iteration from 20 to 5 (with adjusted step size) (ablation) | MOTO · double link swing-up | sample efficiency | Mixed |
| Use MOTO instead of REPS and MORE for policy update on table tennis | robot table tennis (DMP baselines) · noiseless and varying context table tennis | convergence speed | Improved |
| Use MOTO instead of REPS and MORE on table tennis | robot table tennis · noiseless and varying context table tennis | final policy return | Hurt |
| Use MOTO's time-dependent linear policy instead of DMP for table tennis with bounce noise | robot table tennis · noisy ball bounce | adaptation to noise | Improved |
Closest papers by meaning
- Model-Free Trajectory-based Policy Optimization with Monotonic Improvement (JMLR 2018 · similarity 0.94)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.88)
- QPRL : Learning Optimal Policies with Quasi-Potential Functions for Asymmetric Traversal (ICML 2025 · similarity 0.87)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
- Reinforcement Learning with Random Time Horizons (ICML 2025 · similarity 0.86)
- Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
