Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Fast Population-Based Reinforcement Learning on a Single Machine
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Vectorize population updates with Jax | SAC and TD3 training on MuJoCo environments · HalfCheetah-v2 | update-step speedup: at least 4 | Helped |
| Compile the static computation graph | Population-based SAC, TD3, and DQN updates · HalfCheetah-v2 and Atari 2600 | update-step speedup: 2 to 14 | Helped |
| Use Jax vectorization instead of per-agent parallel processes | Population-based SAC, TD3, and DQN · HalfCheetah-v2 and Atari 2600 | update-step time | Helped |
| Increase population size when using JIT compilation | Vectorized population training · HalfCheetah-v2 | speed advantage from compilation | Hurt |
| Vectorize updates on a single CPU core | Population-based training · CPU implementation | update-step speed | No clear effect |
| Use Jax rather than PyTorch on CPU for Atari DQN | DQN · Atari 2600 | update-step speed | Hurt |
| Train 80 agents with PBT instead of a single agent | TD3 and SAC · Gym locomotion environments | episode return over wall time | Mixed |
| Tune hyperparameters with PBT | TD3 and SAC · Gym locomotion environments | asymptotic performance | Mixed |
| Replace sequential shared-critic update ordering with population-batched critic updates (ablation) | TD3 agents with shared critic parameters · MuJoCo locomotion environments | asymptotic performance and sample efficiency | No clear effect |
| Use vectorized Jax updates instead of the original sequential implementation | CEM-RL · MuJoCo environments | runtime at comparable performance | Helped |
Closest papers by meaning
- Effective Diversity in Population Based Reinforcement Learning (NeurIPS 2020 · similarity 0.90)
- Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits (NeurIPS 2020 · similarity 0.90)
- Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning (ICLR 2020 · similarity 0.88)
- Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL (NeurIPS 2021 · similarity 0.88)
- Enhancing Diversity In Parallel Agents: A Maximum State Entropy Exploration Story (ICML 2025 · similarity 0.87)
- Enabling Adaptive Agent Training in Open-Ended Simulators by Targeting Diversity (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.