Research map / Reinforcement learning / Visual and representation RL
Hierarchies of Reward Machines
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a non-flat HRM instead of an equivalent flat HRM | LHRM on CraftWorld MilkBucket · CraftWorld MilkBucket (OP) | HRM learning time: 3.2~(0.6) -> 1.5~(0.2) | Helped |
| Use LHRM instead of DeepSynth to learn an equivalent flat HRM | CraftWorld MilkBucket · CraftWorld MilkBucket (OP) | HRM learning time: 325.6~(29.7) -> 3.2~(0.6) | Helped |
| Use LHRM instead of JIRP to learn an equivalent flat HRM | CraftWorld MilkBucket · CraftWorld MilkBucket (OP) | HRM learning time: 17.1~(5.5) -> 3.2~(0.6) | Helped |
| Use LHRM instead of LRM to learn an equivalent flat HRM | CraftWorld MilkBucket · CraftWorld MilkBucket (OP) | HRM learning time: 347.5~(64.5) -> 3.2~(0.6) | Helped |
| Use options rather than primitive actions to explore for goal traces | LHRM · CraftWorld FRL MilkBucket | episodes to collect goal traces: 128.1\texttimes more episodes | Helped |
| Use options rather than primitive actions to explore for goal traces | LHRM · CraftWorld OPL | episodes to collect goal traces: 53.1\texttimes more episodes | Helped |
| Use options rather than primitive actions to explore for goal traces | LHRM · WaterWorld WD | episodes to collect goal traces: 10.1\texttimes more episodes | Helped |
| Restrict callable RMs to those required by the target HRM (ablation) | LHRM HRM learning · CraftWorld | HRM learning time: 5-7\texttimes faster | Helped |
| Restrict callable RMs to those required by the target HRM (ablation) | LHRM HRM learning · CraftWorld | calls to learner: 20\% fewer calls | Helped |
| Restrict callable RMs to those required by the target HRM (ablation) | LHRM HRM learning · WaterWorld | HRM learning time: 1.5\texttimes faster | Helped |
| Use only a single goal trace to learn the first HRMs (ablation) | LHRM · CraftWorld | HRM learning feasibility | Hurt |
| Use a non-flat rather than a flat HRM for policy learning | HRL policy learning · CraftWorld FRL, hardest tasks | convergence rate | Helped |
Closest papers by meaning
- Contextual Pre-planning on Reward Machine Abstractions for Enhanced Transfer in Deep Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Reward Machines for Deep RL in Noisy and Uncertain Environments (NeurIPS 2024 · similarity 0.89)
- Discriminative Policy Optimization for Token-Level Reward Models (ICML 2025 · similarity 0.87)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- RM-R1: Reward Modeling as Reasoning (ICLR 2026 · similarity 0.87)
- Reward-Respecting Subtasks for Model-Based Reinforcement Learning (Abstract Reprint) (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.