Research map / Reinforcement learning / MDPs and planning
Generalized Emphatic Temporal Difference Learning: Bias-Variance Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ETD(0, β) instead of standard TD | Off-policy value-function approximation in the 2-state MDP · 2-state MDP numerical illustration | mean-squared error: smaller by two orders of magnitude | Helped |
| Set β=0.8 rather than β=γ=0.9 (ablation) | ETD(0, β) · 2-state MDP numerical illustration | mean-squared error | Helped |
| Increase β (ablation) | ETD(0, β) · Off-policy value estimation | bias | Helped |
| Increase β (ablation) | ETD(0, β) · Off-policy value estimation | variance of the emphatic trace: β ≥ 1/√λ(μ,π) | Hurt |
| Use β satisfying 2β² ≥ 1 in the two-state variance example (ablation) | ETD(0, β) · Two-state variance example | variance of F_t: 2β² ≥ 1 | Hurt |
| Increase λ toward 1 (ablation) | ETD(λ, β) · Contraction analysis | contraction modulus: λ → 1 | Helped |
Closest papers by meaning
- PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method (ICLR 2022 · similarity 0.93)
- A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning (NeurIPS 2022 · similarity 0.88)
- Bridging the Gap Between Average and Discounted TD Learning (ICML 2026 · similarity 0.88)
- Temporal Difference Learning as Gradient Splitting (ICML 2021 · similarity 0.87)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.87)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.