Research map / Reinforcement learning / Visual and representation RL
Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX
Waris Radji, Thomas Michel, Hector Piteau
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a higher learning rate (5e-4 or 1e-3 instead of 2.5e-4) for PPO (ablation) | PPO on Octax Pong · Pong grid search | final eval score: -3.20 -> -2.34 | Improved |
| increase parallel environments from 512 to 1024 (ablation) | PPO on Octax Pong · Pong grid search | final eval score: -2.34 -> -4.34 | Hurt |
| use shorter rollouts (32 steps instead of longer) (ablation) | PPO on Octax Pong · Pong grid search | final eval score: -4.17 (64 steps) vs -3.20 (32 steps) | Improved |
| run environments with JAX on GPU instead of CPU EnvPool | Octax vs EnvPool ALE Pong · Pong throughput | environment steps per second: 25,000 -> 350,000 (14x) | Improved |
| ask gpt-4o-mini to recover termination logic instead of score logic | LLM-assisted reward extraction · 21 CHIP-8 games | perfect match rate: 57% -> 19% | Hurt |
| increase LLM-generated game difficulty level (1 to 3) | PPO on Target Shooter · Target Shooter levels | final return: 10.0 -> 8.0 | Hurt |
Closest papers by meaning
- JaxMARL: Multi-Agent RL Environments and Algorithms in JAX (NeurIPS 2024 · similarity 0.86)
- Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX (ICLR 2024 · similarity 0.86)
- Using Unity to Help Solve Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning (ICML 2024 · similarity 0.84)
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (ICLR 2025 · similarity 0.83)
- XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
