Research map / Reinforcement learning / Visual and representation RL
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, Koray Kavukcuoglu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add IMPALA multi-task training with PBT rather than training separate experts | IMPALA deep on DMLab-30 · DMLab-30 test | mean capped human normalised score: 44.5% -> 49.4% | Improved |
| Add population-based training (ablation) | IMPALA deep on DMLab-30 · DMLab-30 test | mean capped human normalised score: 46.5% -> 49.4% | Improved |
| Use the deep residual architecture instead of the shallow architecture (ablation) | IMPALA on DMLab-30 · DMLab-30 test | mean capped human normalised score: 37.1% -> 46.5% | Improved |
| Use 8 learner GPUs rather than 1 learner GPU (ablation) | IMPALA deep, PBT on DMLab-30 · DMLab-30 test | mean capped human normalised score: 49.4% -> 49.1% | No clear effect |
| Use a deep residual architecture instead of a shallow network (ablation) | IMPALA experts on Atari-57 · Atari-57 | median human normalised return: 93.2% -> 191.8% | Improved |
| Train one shared agent across all 57 games instead of separate experts (ablation) | IMPALA deep on Atari-57 · Atari-57 | median human normalised return: 191.8% -> 59.7% | Hurt |
| Use IMPALA rather than A3C for deep Atari experts | Deep expert agents · Atari-57 | median human normalised return: 117.9% -> 191.8% | Improved |
| Add IMPALA's distributed actor-learner architecture rather than use A3C | DeepMind Lab task 1 · seekavoid_arena_01 | frames per second: 6.5K -> 21K | Improved |
| Add IMPALA throughput optimisations and batch size 128 (ablation) | Distributed IMPALA · Computational Performance | frames per second: 80K -> 250K | Improved |
| Add V-trace traces rather than use 1-step importance sampling (ablation) | Off-policy actor-critic with replay · lasertag_three_opponents_small | average final return: 26.4 -> 34.5 | Improved |
| Add V-trace traces rather than use 1-step importance sampling (ablation) | Off-policy actor-critic with replay · explore_goal_locations_small | average final return: 204.8 -> 250.8 | Improved |
| Add V-trace traces rather than use 1-step importance sampling (ablation) | Off-policy actor-critic with replay · seekavoid_arena_01 | average final return: 41.6 -> 46.9 | Improved |
Closest papers by meaning
- Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platform (ICLR 2024 · similarity 0.87)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.85)
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (ICLR 2025 · similarity 0.84)
- Offline Actor-Critic Reinforcement Learning Scales to Large Models (ICML 2024 · similarity 0.84)
- D-VLA: A High-Concurrency Distributed Asynchronous RL Framework for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.84)
- Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
