Research map / Reinforcement learning / MDPs and planning
Finite Sample Analysis of the GTD Policy Evaluation Algorithms in Markov Setting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Analyze GTD under Markov-generated data rather than assuming i.i.d. samples | GTD policy evaluation with linear function approximation · Markov setting | finite-sample estimation error | Helped |
| Use a Markov process with a smaller mixing time | GTD algorithms · Markov setting | convergence rate | Helped |
| Add experience replay to improve the data process's mixing property | GTD algorithms · Markov-generated data | primal-dual gap error | Helped |
| Use a step size α_t = O(1/√t) | GTD algorithms · Markov setting | convergence rate: O(ln(T)/√T) | Helped |
| Use a step size α_t = O(1/t) | GTD algorithms · Markov setting | convergence rate: O(1/ln(T)) | Helped |
| Use a constant step size α_t = O(1/√T) | GTD algorithms · Markov setting | convergence rate: O(1/√T) | Helped |
| Vary the step-size schedule among constant, O(1/√t), and O(1/t) | Gradient algorithm for a convex-concave saddle problem · Simulation | primal-dual gap error | Mixed |
| Generate samples from a Markov process with a smaller mixing time | Gradient algorithm for a convex-concave saddle problem · Simulation | primal-dual gap error | Helped |
| Generate data i.i.d. from the stationary distribution instead of from a Markov chain | Gradient algorithm for a convex-concave saddle problem · Simulation | primal-dual gap error | Helped |
Closest papers by meaning
- Stochastic Variance Reduction Methods for Policy Evaluation (ICML 2017 · similarity 0.88)
- Policy Evaluation for Variance in Average Reward Reinforcement Learning (ICML 2024 · similarity 0.87)
- Beyond Expected Return: Accounting for Policy Reproducibility When Evaluating Reinforcement Learning Algorithms (AAAI 2024 · similarity 0.87)
- Generalizable Policy Improvement via Reinforcement Sampling (Student Abstract) (AAAI 2024 · similarity 0.87)
- Doubly Optimal Policy Evaluation for Reinforcement Learning (ICLR 2025 · similarity 0.86)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.