Research map / Reinforcement learning / MDPs and planning
Learning the Variance of the Reward-To-Go
Aviv Tamar, Dotan Di Castro, Shie Mannor
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LSTD(0) variance estimation with function approximation instead of Monte Carlo | LSTD(0) with RBF and tile features · American put option policy evaluation | RMS error of standard deviation of reward-to-go | Improved |
| increase sample budget N toward large values | LSTD(0) vs Monte Carlo · American put option policy evaluation | RMS error of standard deviation of reward-to-go | Mixed |
| vary lambda in LSTD(λ) (0, 0.3, 0.7, 0.9) (ablation) | LSTD(λ) with tile features · American put option policy evaluation | RMS error of standard deviation of reward-to-go | Mixed |
| replace LSTD(λ) with direct least squares regression (ablation) | LSTD(λ) · American put option policy evaluation | RMS error of standard deviation of reward-to-go | No clear effect |
Closest papers by meaning
- Policy Evaluation for Variance in Average Reward Reinforcement Learning (ICML 2024 · similarity 0.88)
- Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes (AAAI 2024 · similarity 0.86)
- Achieving Tractable Minimax Optimal Regret in Average Reward MDPs (NeurIPS 2024 · similarity 0.86)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.85)
- Planning and Learning in Average Risk-aware MDPs (NeurIPS 2025 · similarity 0.85)
- Risk Estimation in a Markov Cost Process: Lower and Upper Bounds (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
