AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Accelerating Reinforcement Learning through GPU Atari Emulation

Steven Dalton, iuri frosio

NeurIPS 2020 · Reinforcement learning › Visual and representation RL · approach: Scalable RL systems and environments · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace CPU Atari emulation with GPU-parallel CuLE emulationAtari inference · Atari suite, 4096 environmentsFPS: 18K -> 56KImproved
Run GPU emulation with 4096 environmentsAtari emulation · Atari suiteFPS: 18K -> 64KImproved
Use 1200 CuLE environments with one batch and N=5 (ablation)A2C+V-trace · Pongtime to score 18Hurt
Use 5 batches with V-trace for off-policy correction (ablation)A2C+V-trace with 120 OpenAI Gym environments · Atari trainingconvergence timeHurt
Use 5 batches with V-trace (ablation)A2C+V-trace with CuLE and 1200 environments · Pongtime to score 18: 21.2 -> 5.9Improved
Use 5 batches with V-trace (ablation)A2C+V-trace with CuLE and 1200 environments · MsPacmantime to score 1,500: 16.6 -> 6.9Improved
Extend N-step bootstrapping from 5 to 20 (ablation)A2C+V-trace with CuLE · Pongtime to score 18: 5.9 -> 3.1Improved
Extend N-step bootstrapping from 5 to 20 (ablation)A2C+V-trace with CuLE · MsPacmantime to score 1,500: 6.9 -> 11.8Hurt
Scale CuLE batched training from one GPU to four GPUs (ablation)A2C+V-trace · Pongtime to score 18: 3.1 -> 2.4Improved
Scale CuLE batched training from one GPU to four GPUs (ablation)A2C+V-trace · Assaulttraining FPS: 11.0 -> 42.7Improved
Increase environment count from 1024 to 4096 (ablation)CuLE · Atari emulationFPS per environmentMixed
Split Atari CPU and TIA work into separate CUDA kernels (ablation)CuLE emulator · Atari emulationGPU usageImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy