Research map / Reinforcement learning / MDPs and planning
Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Quantum Natural Policy Gradient (QNPG), a deterministic truncated NPG estimator with quantum mean-estimation inner loop, achieving O~(eps^-1.5) quantum oracle sample complexity for infinite-horizon MDPs with general parameterized policies.
Problem
Model-free reinforcement learning with general parameterized policies in infinite-horizon MDPs under quantum oracle access; classical NPG sample complexity is O~(eps^-2), and quantum settings require deterministic (non-geometric-random-length) gradient estimators.
Approach
Replace random geometric-length trajectory sampling in NPG with deterministic truncated gradient and Fisher estimators computed over superpositions of fixed-length trajectories, with bias decaying as gamma^N, and apply quantum variance reduction (QVarianceReduce) in the inner SGD loop while keeping the outer loop classical.
Key results
- QNPG achieves O~(eps^-1.5) sample complexity in quantum oracle queries versus classical O~(eps^-2) for MDP queries, with iteration complexity O(eps^-1).
- Truncation bias decays as O(gamma^N), so N and H need only be O(log(1/eps)).
- Optimality gap bounded by sqrt(eps_bias)+eps under compatible function approximation and Fisher non-degeneracy assumptions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QNPG with quantum variance reduction in the inner loop and deterministic truncated estimators | Parameterized infinite-horizon MDPs · Parameterized infinite-horizon MDP | sample complexity: \tilde{\mathcal{O}}(\epsilon^{-2}) -> \tilde{\mathcal{O}}(\epsilon^{-1.5}) | Helped |
| Truncate trajectory-based gradient and Fisher estimators at N steps | QNPG · Policy gradient and Fisher estimation | estimator bias: \cO(\gamma^N) | Mixed |
Limitations
- Theoretical only; no empirical experiments on quantum hardware or simulation.
- Requires strong assumptions (score function Lipschitz/smooth, Fisher non-degeneracy, compatible approximation error).
- Outer loop remains classical, so speedup limited to inner loop; quantum actor-critic not addressed.
Experimental setup
Metrics: sample complexity optimality gap
Baselines: classical NPG sample complexity O(eps^-2) classical policy gradient methods
Builds on: Natural Policy Gradient quantum mean estimation quantum stochastic convex optimization QVarianceReduce
Closest papers by meaning
- Benchmarking Quantum Reinforcement Learning (ICML 2025 · similarity 0.88)
- Quantum Algorithms for Finite-horizon Markov Decision Processes (ICML 2025 · similarity 0.87)
- Quantum Speedups in Regret Analysis of Infinite Horizon Average-Reward Markov Decision Processes (ICML 2025 · similarity 0.87)
- Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret (ICML 2024 · similarity 0.87)
- TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search (NeurIPS 2025 · similarity 0.86)
- Reinforcement Learning for Quantum Control under Physical Constraints (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.