Research map / Reinforcement learning / Visual and representation RL
Scalable Option Learning in High-Throughput Environments
Mikael Henaff, Scott Fujimoto, Michael Matthews, Michael Rabbat
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use batched single-network hierarchical system (SOL) instead of prior hierarchical implementations | hierarchical RL · NLE throughput | steps/second: 55 -> 43000 | Improved |
| hierarchical SOL vs flat APPO throughput | SOL · NLE throughput | steps/second: 50000 -> 43000 | No clear effect |
| add hierarchy with option rewards (SOL) instead of flat APPO with option rewards | APPO (task+option rewards) · MiniHack TreasureDash | points: 23 -> 28 | Improved |
| add option rewards (Health) to flat APPO task reward | APPO · NetHackScore | game score | No clear effect |
| use SOL hierarchy with task reward only (HiPPO) instead of option rewards (ablation) | SOL+HiPPO · NetHackScore | game score | No clear effect |
| add option rewards and hierarchy together (SOL) over flat agents | SOL · NetHackScore | game score | Improved |
| add Motif LLM rewards to SOL Score option | SOL+Motif · NetHackScore | game score | Improved |
| learn option rewards with DIAYN instead of hand-specified (ablation) | SOL · MiniHack | return | No clear effect |
| use SOL instead of flat APPO on hard G-maze | SOL · PointMaze G-maze | success rate: 0% -> roughly 70% | Improved |
| use SOL on easy U-Maze | SOL · PointMaze U-Maze | success rate | No clear effect |
Closest papers by meaning
- Hierarchical Reinforcement Learning via Advantage-Weighted Information Maximization (ICLR 2019 · similarity 0.88)
- Horizon Reduction Makes RL Scalable (NeurIPS 2025 · similarity 0.86)
- Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoals (ICML 2025 · similarity 0.86)
- Deep Reinforcement Learning from Hierarchical Preference Design (ICML 2025 · similarity 0.86)
- Strategic Planning: A Top-Down Approach to Option Generation (ICML 2025 · similarity 0.85)
- Accelerating Task Generalisation with Multi-Level Skill Hierarchies (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
