Research map / Optimization / Nonconvex and smooth optimization
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use diminishing stepsizes with σ=3/2ν=1 for TDC | Two-time-scale TDC · Markovian samples | training error convergence rate: \mathcal{O}(\frac{\log t}{t^{2/3}}) | Helped |
| Use constant stepsizes for both iterates instead of diminishing stepsizes (ablation) | Two-time-scale TDC · Garnet \mathcal{G}(500,20,50,20) | convergence speed | Mixed |
| Use small constant stepsizes instead of diminishing stepsizes (ablation) | Two-time-scale TDC · Garnet \mathcal{G}(500,20,50,20) | training error | No clear effect |
| Set the fast-timescale decay rate to ν=2/3σ (ablation) | TDC with diminishing stepsize · Garnet \mathcal{G}(500,20,50,20) | error decay rate | Helped |
| Set ν:σ=1:3 for diminishing stepsizes at σ=0.45 or σ=0.6 (ablation) | TDC with diminishing stepsize · Garnet \mathcal{G}(500,20,50,20) | training error | Hurt |
| Use blockwise diminishing stepsizes instead of ordinary diminishing stepsizes (ablation) | TDC · Garnet \mathcal{G}(500,20,50,20) | convergence speed | Helped |
| Use blockwise diminishing stepsizes instead of ordinary diminishing stepsizes (ablation) | TDC · Garnet \mathcal{G}(500,20,50,20) | training error | No clear effect |
| Reduce the ratio of slow to fast stepsizes while using constant stepsizes | Two-time-scale TDC · Markovian samples | training error: α/β→0 | Helped |
Closest papers by meaning
- Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis (NeurIPS 2020 · similarity 0.94)
- Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation (NeurIPS 2021 · similarity 0.93)
- A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning (NeurIPS 2022 · similarity 0.91)
- Towards Parameter-Free Temporal Difference Learning (ICML 2026 · similarity 0.91)
- Revisiting a Design Choice in Gradient Temporal Difference Learning (ICLR 2025 · similarity 0.88)
- Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.