Research map / Optimization / Nonconvex and smooth optimization
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add variance reduction to TD and TDC (VRTD, VRTDC vs TD, TDC) | TD/TDC · Garnet problem | asymptotic convergence error | Helped |
| two time-scale variance reduction (VRTDC) instead of one time-scale (VRTD) | VRTD · Garnet problem | mean convergence error | Helped |
| VRTDC vs VRTD across batch sizes M (ablation) | VRTD · Garnet problem | asymptotic convergence error | Helped |
| add variance reduction to TDC | TDC · Garnet problem | variance of stochastic updates (theta and w) | Helped |
| VRTDC vs VRTD | VRTD · Frozen Lake | asymptotic convergence error | Helped |
| variance reduction on one time-scale only (VRTD) vs TDC | TDC · Frozen Lake | asymptotic convergence error | No clear effect |
| vary batch size M for VRTDC vs VRTD (ablation) | VRTD · Frozen Lake | asymptotic convergence error | Helped |
| add variance reduction to TDC | TDC · Frozen Lake | variance of pseudo-gradient updates | Helped |
Closest papers by meaning
- Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples (NeurIPS 2019 · similarity 0.94)
- Reanalysis of Variance Reduced Temporal Difference Learning (ICLR 2020 · similarity 0.93)
- Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation (NeurIPS 2021 · similarity 0.90)
- Truncated Variance Reduced Value Iteration (NeurIPS 2024 · similarity 0.87)
- Policy Evaluation for Variance in Average Reward Reinforcement Learning (ICML 2024 · similarity 0.86)
- Statistical Efficiency of Distributional Temporal Difference Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.