Research map / Reinforcement learning / Visual and representation RL
Accelerating Reinforcement Learning through GPU Atari Emulation
Steven Dalton, iuri frosio
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CPU Atari emulation with GPU-parallel CuLE emulation | Atari inference · Atari suite, 4096 environments | FPS: 18K -> 56K | Improved |
| Run GPU emulation with 4096 environments | Atari emulation · Atari suite | FPS: 18K -> 64K | Improved |
| Use 1200 CuLE environments with one batch and N=5 (ablation) | A2C+V-trace · Pong | time to score 18 | Hurt |
| Use 5 batches with V-trace for off-policy correction (ablation) | A2C+V-trace with 120 OpenAI Gym environments · Atari training | convergence time | Hurt |
| Use 5 batches with V-trace (ablation) | A2C+V-trace with CuLE and 1200 environments · Pong | time to score 18: 21.2 -> 5.9 | Improved |
| Use 5 batches with V-trace (ablation) | A2C+V-trace with CuLE and 1200 environments · MsPacman | time to score 1,500: 16.6 -> 6.9 | Improved |
| Extend N-step bootstrapping from 5 to 20 (ablation) | A2C+V-trace with CuLE · Pong | time to score 18: 5.9 -> 3.1 | Improved |
| Extend N-step bootstrapping from 5 to 20 (ablation) | A2C+V-trace with CuLE · MsPacman | time to score 1,500: 6.9 -> 11.8 | Hurt |
| Scale CuLE batched training from one GPU to four GPUs (ablation) | A2C+V-trace · Pong | time to score 18: 3.1 -> 2.4 | Improved |
| Scale CuLE batched training from one GPU to four GPUs (ablation) | A2C+V-trace · Assault | training FPS: 11.0 -> 42.7 | Improved |
| Increase environment count from 1024 to 4096 (ablation) | CuLE · Atari emulation | FPS per environment | Mixed |
| Split Atari CPU and TIA work into separate CUDA kernels (ablation) | CuLE emulator · Atari emulation | GPU usage | Improved |
Closest papers by meaning
- CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning (ICLR 2026 · similarity 0.89)
- StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning (ICML 2026 · similarity 0.85)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.84)
- Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
- NAVIX: Scaling MiniGrid Environments with JAX (NeurIPS 2025 · similarity 0.83)
- Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
