Research map / Reinforcement learning / Policy gradient and actor-critic
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IQM instead of median to aggregate performance across tasks and runs | Deep RL benchmark evaluation · Atari 100k | confidence interval width | Helped |
| Use IQM instead of median to detect score improvements | Deep RL benchmark evaluation · Atari 100k | runs needed to detect improvement | Helped |
| Use percentile stratified bootstrap confidence intervals for aggregate metrics | Deep RL benchmark evaluation · Atari 100k | confidence interval coverage: 95% | Helped |
| Use run-score distributions instead of average-score distributions | Deep RL benchmark evaluation · Atari 100k | bias and robustness to outliers | Helped |
| Report point estimates without interval estimates | Deep RL benchmark comparisons · Atari 100k | reliability of comparative conclusions | Hurt |
| Use the sample median rather than IQM | Deep RL benchmark evaluation · Atari 100k | bias: 0.05 points vs 0.006 points | Hurt |
| Use maximum evaluation scores during training instead of end-performance | Atari 100k algorithm comparisons · Atari 100k | comparability of reported scores | Hurt |
| Use PPO-normalized scores rather than minimum-maximum normalization | Procgen benchmark evaluation · Procgen easy mode | robustness of aggregate scores to heavy-tailed tasks | Hurt |
| Evaluate algorithm comparisons using uncertainty estimates rather than point estimates alone | Deep RL benchmark comparisons · DeepMind Control Suite | consistency of claimed rankings | Hurt |
| Increase the number of runs for median-based comparisons | Deep RL benchmark evaluation · Atari 100k | uncertainty in median scores: 50--100 runs | Helped |
| Use only 3 runs for stratified bootstrap confidence intervals | Deep RL benchmark evaluation · Atari 100k | 95% confidence interval coverage: 3 runs | Hurt |
| Compare SPR with DrQ(ε) using 5 runs | SPR · Atari 100k | median score improvement: (-0.09, 0.18) | No clear effect |
Closest papers by meaning
- Deep Reinforcement Learning That Matters (AAAI 2018 · similarity 0.90)
- Evaluating the Performance of Reinforcement Learning Algorithms (ICML 2020 · similarity 0.90)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.89)
- Position: Benchmarking is Limited in Reinforcement Learning Research (ICML 2024 · similarity 0.88)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.88)
- The Effective Horizon Explains Deep RL Performance in Stochastic Environments (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.