Research map / Reinforcement learning / Policy gradient and actor-critic
The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QTD(128) instead of TD | Tabular policy evaluation · Stochastic MRPs | mean-squared error | Helped |
| Use QTD(128) instead of TD | Tabular policy evaluation · Deterministic environment | mean-squared error | Hurt |
| Use QTD(128) instead of TD | Tabular policy evaluation · MRPs with t2-distributed rewards | mean-squared error | Helped |
| Increase reward-noise standard deviation | QTD(128) relative to TD · Deterministic cycle with Gaussian rewards | relative mean-squared error | Helped |
| Reduce the number of quantiles to a low value (ablation) | QTD · Example environment with high fixed-point bias | mean-squared error | Hurt |
| Use QTD(1) instead of QTD(128) (ablation) | QTD · Main suite of MRPs | mean-squared error | Mixed |
| Use QTD(16) instead of QTD(128) (ablation) | QTD · Stochastic MRPs in the main suite | mean-squared error | No clear effect |
| Replace QTD targets with one-step mean-value targets (PQTD) (ablation) | QTD(128) · Suite of MRPs | mean-squared error | Hurt |
| Use QTD's averaged quantile updates instead of PQTD's one-step quantile updates (ablation) | PQTD · Cycle environment with Gaussian reward noise | update variance | Helped |
| Use a larger-than-optimal learning rate | QTD relative to TD · Stochastic MRPs | mean-squared error degradation | Helped |
Closest papers by meaning
- Statistical Efficiency of Distributional Temporal Difference Learning (NeurIPS 2024 · similarity 0.89)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.89)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.88)
- Quantile Geometry Regularization for Distributional Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- An Analysis of Quantile Temporal-Difference Learning (ICML 2025 · similarity 0.87)
- A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.