AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, Koray Kavukcuoglu

ICML 2018 · Reinforcement learning › Visual and representation RL · approach: Scalable RL systems and environments · 571 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add IMPALA multi-task training with PBT rather than training separate expertsIMPALA deep on DMLab-30 · DMLab-30 testmean capped human normalised score: 44.5% -> 49.4%Improved
Add population-based training (ablation)IMPALA deep on DMLab-30 · DMLab-30 testmean capped human normalised score: 46.5% -> 49.4%Improved
Use the deep residual architecture instead of the shallow architecture (ablation)IMPALA on DMLab-30 · DMLab-30 testmean capped human normalised score: 37.1% -> 46.5%Improved
Use 8 learner GPUs rather than 1 learner GPU (ablation)IMPALA deep, PBT on DMLab-30 · DMLab-30 testmean capped human normalised score: 49.4% -> 49.1%No clear effect
Use a deep residual architecture instead of a shallow network (ablation)IMPALA experts on Atari-57 · Atari-57median human normalised return: 93.2% -> 191.8%Improved
Train one shared agent across all 57 games instead of separate experts (ablation)IMPALA deep on Atari-57 · Atari-57median human normalised return: 191.8% -> 59.7%Hurt
Use IMPALA rather than A3C for deep Atari expertsDeep expert agents · Atari-57median human normalised return: 117.9% -> 191.8%Improved
Add IMPALA's distributed actor-learner architecture rather than use A3CDeepMind Lab task 1 · seekavoid_arena_01frames per second: 6.5K -> 21KImproved
Add IMPALA throughput optimisations and batch size 128 (ablation)Distributed IMPALA · Computational Performanceframes per second: 80K -> 250KImproved
Add V-trace traces rather than use 1-step importance sampling (ablation)Off-policy actor-critic with replay · lasertag_three_opponents_smallaverage final return: 26.4 -> 34.5Improved
Add V-trace traces rather than use 1-step importance sampling (ablation)Off-policy actor-critic with replay · explore_goal_locations_smallaverage final return: 204.8 -> 250.8Improved
Add V-trace traces rather than use 1-step importance sampling (ablation)Off-policy actor-critic with replay · seekavoid_arena_01average final return: 41.6 -> 46.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy