Research map / Reinforcement learning / Policy gradient and actor-critic
Accelerated Gradient Temporal Difference Learning
Yangchen Pan, Adam White, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use low-rank ATD instead of linear TD baselines | Policy evaluation on Boyan's chain · Boyan's chain | percentage error | Improved |
| Use ATD instead of the tested competing methods | Policy evaluation with tile-coded features · Mountain Car | learning speed | Improved |
| Use ATD instead of tLSTD at small rank | Policy evaluation with tile-coded features · Mountain Car | percentage error | Improved |
| Use ATD instead of tLSTD | Policy evaluation with tile-coded features · Energy allocation | learning speed | No clear effect |
| Use ATD instead of tLSTD with stochastic action selection | Policy evaluation with tile-coded features · Mountain Car | performance | Improved |
| Add 100 noisy features, with 50 randomly set to one (ablation) | ATD and comparison methods · Mountain Car with noisy features | percentage error | Hurt |
| Use ATD's low-rank approximation to ignore unreliable feature information (ablation) | ATD · Mountain Car with noisy features | learning efficiency | Improved |
| Add a low-rank preconditioner to the TD update (ablation) | TD(λ) · General policy evaluation; expected-update analysis | convergence to the TD fixed point | No clear effect |
| Increase the low-rank approximation rank (ablation) | ATD · Mountain Car | convergence speed | Mixed |
| Set ATD rank to 100 (ablation) | ATD · Mountain Car | performance | Hurt |
| Use a diagonal approximation instead of a low-rank approximation (ablation) | ATD · Exploratory experiments | performance | Hurt |
| Use a rank-k approximation while retaining the sampled full TD error update (ablation) | ATD · General policy evaluation; convergence analysis | asymptotic bias | Improved |
Closest papers by meaning
- Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples (NeurIPS 2019 · similarity 0.89)
- Temporal Difference Learning: Why It Can Be Fast and How It Will Be Faster (ICLR 2025 · similarity 0.89)
- Gradient Temporal-Difference Learning with Regularized Corrections (ICML 2020 · similarity 0.89)
- Temporal Difference Learning as Gradient Splitting (ICML 2021 · similarity 0.88)
- Towards Parameter-Free Temporal Difference Learning (ICML 2026 · similarity 0.88)
- Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
