AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Accelerated Gradient Temporal Difference Learning

Yangchen Pan, Adam White, Martha White

AAAI 2017 · Reinforcement learning › Policy gradient and actor-critic · approach: Temporal-difference and discounting theory · 23 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use low-rank ATD instead of linear TD baselinesPolicy evaluation on Boyan's chain · Boyan's chainpercentage errorImproved
Use ATD instead of the tested competing methodsPolicy evaluation with tile-coded features · Mountain Carlearning speedImproved
Use ATD instead of tLSTD at small rankPolicy evaluation with tile-coded features · Mountain Carpercentage errorImproved
Use ATD instead of tLSTDPolicy evaluation with tile-coded features · Energy allocationlearning speedNo clear effect
Use ATD instead of tLSTD with stochastic action selectionPolicy evaluation with tile-coded features · Mountain CarperformanceImproved
Add 100 noisy features, with 50 randomly set to one (ablation)ATD and comparison methods · Mountain Car with noisy featurespercentage errorHurt
Use ATD's low-rank approximation to ignore unreliable feature information (ablation)ATD · Mountain Car with noisy featureslearning efficiencyImproved
Add a low-rank preconditioner to the TD update (ablation)TD(λ) · General policy evaluation; expected-update analysisconvergence to the TD fixed pointNo clear effect
Increase the low-rank approximation rank (ablation)ATD · Mountain Carconvergence speedMixed
Set ATD rank to 100 (ablation)ATD · Mountain CarperformanceHurt
Use a diagonal approximation instead of a low-rank approximation (ablation)ATD · Exploratory experimentsperformanceHurt
Use a rank-k approximation while retaining the sampled full TD error update (ablation)ATD · General policy evaluation; convergence analysisasymptotic biasImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy