Research map / Reinforcement learning / MDPs and planning
Approximate Newton Methods for Policy Search in Markov Decision Processes
Thomas Furmston, Guy Lever, David Barber
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use second Gauss-Newton method instead of EM | policy search · Ball-in-a-cup robot arm | successful runs out of 50: 36 -> 45 | Improved |
| use second Gauss-Newton method instead of natural gradient ascent | policy search · 3-link rigid manipulator | normalized total expected reward: natural gradient ascent obtained around 15% of the performance | Improved |
| use second Gauss-Newton method instead of EM | policy search · 3-link rigid manipulator | normalized total expected reward: EM obtained 40% of the performance; 50 seconds vs 300 seconds | Improved |
| tune step size in Gauss-Newton rather than constant step 1.0 as in EM (ablation) | second Gauss-Newton method · 3-link rigid manipulator | normalized total expected reward | Improved |
| use gradient ascent | policy search · 3-link rigid manipulator | normalized total expected reward | Hurt |
| use full Gauss-Newton instead of diagonal Gauss-Newton (ablation) | Gauss-Newton policy search · Tetris 10x10 board | completed lines | Improved |
| use diagonal Gauss-Newton instead of natural gradient ascent | policy search · Tetris 10x10 board | completed lines | No clear effect |
| use second Gauss-Newton method instead of EM | policy search · Ball-in-a-cup robot arm | total expected reward | Improved |
| use first Gauss-Newton method, gradient ascent or natural gradient ascent | policy search · Ball-in-a-cup robot arm | task success | Hurt |
Closest papers by meaning
- Deterministic Policy Gradient Primal-Dual Methods for Continuous-Space Constrained MDPs (AAAI 2025 · similarity 0.89)
- Beyond Stationarity: Convergence Analysis of Stochastic Softmax Policy Gradient Methods (ICLR 2024 · similarity 0.89)
- Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs (NeurIPS 2026 · similarity 0.88)
- A Single-Loop Robust Policy Gradient Method for Robust Markov Decision Processes (ICML 2024 · similarity 0.88)
- Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity (ICLR 2024 · similarity 0.88)
- Accelerated Policy Gradient for s-rectangular Robust MDPs with Large State Spaces (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
