Research map / Reinforcement learning / Visual and representation RL
Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning
Sotetsu Koyamada, Shinri Okano, Soichiro Nishimori, Yu Murata, Keigo Habara, Haruka Kita, Shin Ishii
This paper tried a common change: Larger batch size
Increase self-play batch size and GPU count eightfold · Go 9x9 · time to reach same performance: approximately four times faster Improved
What 785 accepted papers report for this change
- 469 improved
- 107 no clear effect
- 48 mixed
- 161 hurt
- Predicting the Emergence of Induction Heads in Language Model Pretraining ICML 2026 · increase batch size (fixed context 1024) · Natural data (CC100) · induction head emergence point (updates): rho=-1.0, p<0.001Improved
- When RL Meets Adaptive Speculative Training: A Unified Training-Serving System ICML 2026 · Increase batch size from 1 to 32 · MiniMax M2.1 FP8 · speedup: 1.57x -> 1.25xHurt
- Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking ICML 2026 · increase per-device batch size to 10 using freed memory · Llama 3-8B, 128K · throughput (TPS): 2464.10 (Ulysses) -> 2526.77 (UPipe)Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Run vectorized game simulation on one A100 GPU | Pgx · Tic-tac-toe, Connect Four, chess, Go | simulation throughput vs. Python libraries: at least 10x faster | Improved |
| Scale simulation from one to eight A100 GPUs | Pgx · All Pgx environments | simulation throughput: average of 7.4x across all environments; at least 6.6x in the MinAtar Breakout environment | Improved |
| Increase self-play batch size and GPU count eightfold | Gumbel AlphaZero with Pgx · Go 9x9 | time to reach same performance: approximately four times faster | Improved |
| Train a Pgx baseline model with Gumbel AlphaZero | Pgx baseline model · Go 9x9 vs. Pachi | match record: 62 wins and 38 losses out of 100 matches | Improved |
Closest papers by meaning
- JaxMARL: Multi-Agent RL Environments and Algorithms in JAX (NeurIPS 2024 · similarity 0.86)
- Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (ICML 2024 · similarity 0.86)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.86)
- NAVIX: Scaling MiniGrid Environments with JAX (NeurIPS 2025 · similarity 0.85)
- Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
