Research map / Optimization / Nonconvex and smooth optimization
Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a non-adaptive geometrically increasing step size with Q-function compatible approximation | Q-NPG with log-linear policies · Infinite-horizon discounted MDPs | optimality-gap convergence rate: linear convergence up to an error floor, compared with ${\cal O}(1/\sqrt{k}) | Helped |
| Use a non-adaptive geometrically increasing step size with advantage-function compatible approximation | NPG with log-linear policies · Infinite-horizon discounted MDPs | optimality-gap convergence rate: linear convergence up to an error floor | Helped |
| Use an arbitrary constant step size instead of a diminishing step size | Q-NPG · Infinite-horizon discounted MDPs | average optimality-gap convergence rate: ${\cal O}(1/k)$, as opposed to ${\cal O}(1/\sqrt{k}) | Helped |
| Use an arbitrary constant step size instead of a diminishing step size | NPG · Infinite-horizon discounted MDPs | average optimality-gap convergence rate: ${\cal O}(1/k)$ to ${\cal O}(1/\sqrt{k}) | Helped |
| Use sampled Q-NPG with an SGD regression solver | Q-NPG · Infinite-horizon discounted MDPs | sample complexity: ${\cal O} \bigl(\frac{1}{(1-\gamma)^{11}\epsilon^6}\bigr) -> \tilde{{\cal O}} \bigl(\frac{1}{(1-\gamma)^7\epsilon^2}\big | Helped |
| Use sampled NPG with an SGD regression solver | NPG · Infinite-horizon discounted MDPs | sample complexity: ${\cal O} \bigl(\frac{1}{(1-\gamma)^{11}\epsilon^6}\bigr) -> \tilde{{\cal O}} \bigl(\frac{1}{(1-\gamma)^{5}\epsilon^2}\b | Helped |
| Use Q-function regression rather than advantage-function regression | Compatible function approximation for NPG · Log-linear policies | regression computation cost: |{ cal A}| times cheaper | Helped |
Closest papers by meaning
- Sharp Convergence and Sample Complexity of Policy Mirror Descent for Average-Reward MDPs (NeurIPS 2026 · similarity 0.92)
- Convergence of Policy Mirror Descent Beyond Compatible Function Approximation (ICML 2025 · similarity 0.90)
- Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework (NeurIPS 2026 · similarity 0.89)
- Policy Mirror Descent with Lookahead (NeurIPS 2024 · similarity 0.89)
- Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs (NeurIPS 2026 · similarity 0.88)
- Mirror descent actor-critic methods for entropy regularised MDPs in general spaces: stability and convergence (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.