Research map / Optimization / Nonconvex and smooth optimization
Finite Sample Analyses for TD(0) With Function Approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TD(0) with α_n = (n + 1)^{-σ} | Linear TD(0) policy evaluation · Linear policy evaluation (iid samples) | expected squared parameter error: K_1 e^{- (λ/2) n^{1 - σ}} + K_2/n^σ | Helped |
| Use the unaltered TD(0) update with α_n = (n+1)^{-1} | Linear TD(0) policy evaluation · Linear policy evaluation (iid samples) | high-probability sample complexity: \tilde{O}\left(\max\left\{\left[\frac{1}{\epsilon}\right]^{1+ \frac{1}{\lambda}} \left[\ln\frac{1}{\delta}\right]^{1 + \ | Helped |
| Use α_n = (n+1)^{-1} | Linear TD(0) policy evaluation · Linear policy evaluation (iid samples) | high-probability parameter error rate: \tilde{O}\left(n^{-\min\{1/2, \lambda/ (\lambda + 1)\}}\right) | Helped |
| Choose σ closer to 0, giving more slowly decaying and initially larger stepsizes | TD(0) expected convergence · Linear policy evaluation (iid samples) | noiseless-process convergence term: e^{- (λ/2) n^{1 - σ}} | Helped |
| Choose σ closer to 1, giving more quickly decaying stepsizes | TD(0) expected convergence · Linear policy evaluation (iid samples) | noise term in expected squared error: K_2/n^σ | Helped |
| Increase the driving matrix eigenvalue parameter λ | TD(0) high-probability convergence · Linear policy evaluation (iid samples) | high-probability parameter error rate: n^{-\min\{1/2, \lambda/ (\lambda + 1)\}} | Helped |
| Use a larger stepsize by choosing σ smaller, beyond the noise-limited threshold | TD(0) expected convergence · Linear policy evaluation (iid samples) | expected squared parameter error: K_1 e^{- (λ/2) n^{1 - σ}} + K_2/n^σ | Mixed |
| Allow nonsquare-summable stepsizes, including 1/n^κ with κ ∈ (0, 1/2] | TD(0) expected convergence analysis · Linear policy evaluation (iid samples) | applicable stepsize sequences: κ ∈ (0, 1/2] | Helped |
Closest papers by meaning
- Towards Parameter-Free Temporal Difference Learning (ICML 2026 · similarity 0.93)
- Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2025 · similarity 0.91)
- Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2026 · similarity 0.91)
- A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation (NeurIPS 2025 · similarity 0.90)
- An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks (ICML 2024 · similarity 0.89)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.