Research map / Reinforcement learning / MDPs and planning
Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulator
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use self-normalized martingale inequalities in LSTD-Q analysis, reducing the exploration-noise dependence from 1/ση^8 to 1/ση^4 | LSTD-Q for adaptive LQR · Adaptive LQR | regret: T^{4/5} -> T^{2/3} | Helped |
| Use approximate policy iteration with the paper's greedy exploration schedule | Adaptive LQR · Adaptive LQR | regret: T^{2/3+ε} -> T^{2/3} | Helped |
| Allocate samples to policy evaluation across approximate policy iteration steps | LSPI · LQR offline policy learning | sample complexity: (n+d)^3 ε^{-2} per evaluation step; O(log(1/ε)) improvement steps | Mixed |
| Collect fresh trajectory data for each policy evaluation instead of reusing one dataset | LSPI · LQR offline policy learning | analysis tractability | Helped |
| Use adaptive LSPI instead of model-based certainty-equivalence control | Adaptive LQR · Adaptive LQR experiments | regret | Hurt |
| Use LSPI rather than MFLQ | Adaptive LQR · Adaptive LQR experiments | regret | Mixed |
| Use derivative-free optimization rather than policy iteration | Offline LQR experiments · Offline LQR | relative cost error | Mixed |
| Add a value-function baseline to policy gradients | Policy gradients · Offline LQR | estimator variance | Helped |
Closest papers by meaning
- Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States (ICML 2024 · similarity 0.88)
- Polynomial-Time Approximability of Constrained Reinforcement Learning (ICML 2025 · similarity 0.87)
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.87)
- A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning (NeurIPS 2025 · similarity 0.87)
- Sharp Convergence and Sample Complexity of Policy Mirror Descent for Average-Reward MDPs (NeurIPS 2026 · similarity 0.87)
- Computational Hardness of Reinforcement Learning with Partial $q^{\pi}$-Realizability (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.