Research map / Reinforcement learning / Visual and representation RL
Leveraging Procedural Generation to Benchmark Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on a fixed sequence of levels rather than resampling levels across episodes (ablation) | PPO in Procgen environments · Procgen hard generalization | test performance | Hurt |
| Increase training set size from small sets toward as many as 10,000 levels (ablation) | PPO in Procgen environments · Procgen held-out levels | generalization performance | Helped |
| Increase training set size beyond a threshold (ablation) | PPO in Procgen environments · Procgen training levels | training performance | Helped |
| Scale IMPALA convolutional channels from 1x toward 2x or 4x (ablation) | IMPALA policy · Procgen sample efficiency | performance | Helped |
| Scale IMPALA convolutional channels from 1x toward 2x or 4x (ablation) | IMPALA policy · Procgen generalization | test performance | Helped |
| Use the smaller Nature-CNN architecture instead of IMPALA (ablation) | PPO policy · Procgen environments | training performance | Hurt |
| Add a frame stack of 4 instead of using only the current frame (ablation) | PPO baseline · Procgen environments | performance | No clear effect |
| Add an LSTM on top of the convolutional network (ablation) | PPO baseline · Procgen environments | performance | Mixed |
| Use Rainbow instead of PPO | Procgen agents trained and tested on the full level distribution · Procgen Benchmark | performance across environments | Mixed |
| Use 8 parallel workers with shared parameters and averaged gradients (ablation) | Rainbow · Procgen environments | learning performance | Helped |
| Increase batch size and decrease update frequency each by a factor of 16, and increase learning rate by a factor of 4 (ablation) | Rainbow · Procgen environments | performance | No clear effect |
| Remove frame stacking (ablation) | PPO baseline · Procgen environments | performance | No clear effect |
Closest papers by meaning
- Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds (NeurIPS 2022 · similarity 0.88)
- Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (ICML 2024 · similarity 0.88)
- Implicit Curriculum in Procgen Made Explicit (NeurIPS 2024 · similarity 0.87)
- FactorSim: Generative Simulation via Factorized Representation (NeurIPS 2024 · similarity 0.87)
- Position: Benchmarking is Limited in Reinforcement Learning Research (ICML 2024 · similarity 0.87)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.