Research map / Optimization / Nonconvex and smooth optimization
Temporal Difference Learning as Gradient Splitting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Interpret the expected TD(0) update as a gradient splitting and use the combined value and Dirichlet error | Projected TD(0), averaged iterates · Markov observations | expected combined value and Dirichlet error: \frac{ \| \theta^*-\theta_{0}\|_2^2 + G^2\left[9 + 12\tau ^{{\rm mix}}\left(1/\sqrt{T}\right) \right]}{2\sqrt{T}}; prior | Helped |
| Add a separate estimate of the value-function mean and shift the averaged TD(0) output to match it | TD(0) · Policy evaluation with linear function approximation | expected squared value error: \frac{ r_{\rm max}^2 \tau^{{\rm mix}}\left(\frac{1}{2(T+1)}\right)}{(1-\gamma)^2 T} + \frac{ \| \theta^* -\theta_{0}\|_2 | Helped |
| Estimate the value-function mean from sampled rewards | Value-function estimation · Mean value estimation | worst-case mean squared error: \Omega \left( (1/(1-\gamma)^2)/T \right) | Hurt |
Closest papers by meaning
- Towards Parameter-Free Temporal Difference Learning (ICML 2026 · similarity 0.92)
- A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning (NeurIPS 2022 · similarity 0.91)
- Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2026 · similarity 0.90)
- An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks (ICML 2024 · similarity 0.89)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.88)
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.