Research map / Reinforcement learning / Visual and representation RL
Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compose a sequence of learned programs with a meta-policy rather than search for one program embedding | LEAPS-style programmatic RL · Karel | mean return | Helped |
| compose programs to synthesize longer behaviors | LEAPS · Program reconstruction Len 100 | reconstruction performance: 0.13 (0.08) -> 0.26 (0.02) | Helped |
| use the proposed program-generation dataset instead of the original dataset | LEAPS · Karel | mean return | Helped |
| use PPO rather than SAC for meta-policy training (ablation) | HPRL · Karel and Karel-Hard | mean return | Helped |
| use dense per-program rewards rather than reward only after all programs execute (ablation) | HPRL · CleanHouse and Seeder | sample efficiency | Helped |
| optimize program behavior reconstruction loss (ablation) | program embedding learning · Karel | task performance | No clear effect |
| use 16-dimensional rather than 64-dimensional program embeddings (ablation) | HPRL · CleanHouse | mean return: 1.00 (0.00) -> 0.47 (0.06) | Hurt |
| use 16-dimensional rather than 64-dimensional program embeddings (ablation) | HPRL · Seeder | mean return: 0.58 (0.07) -> 0.21 (0.02) | Hurt |
| use 32-dimensional rather than 64-dimensional program embeddings (ablation) | HPRL · CleanHouse | mean return: 1.00 (0.00) -> 0.84 (0.27) | Hurt |
| use 32-dimensional rather than 64-dimensional program embeddings (ablation) | HPRL · Seeder | mean return: 0.58 (0.07) -> 0.35 (0.16) | Hurt |
| increase embedding dimensionality from 64 to 128 (ablation) | HPRL · CleanHouse | mean return: 1.00 (0.00) -> 1.00 (0.00) | No clear effect |
| increase embedding dimensionality from 64 to 128 (ablation) | HPRL · Seeder | mean return: 0.58 (0.07) -> 0.57 (0.03) | No clear effect |
Closest papers by meaning
- Learning to Synthesize Programs as Interpretable and Generalizable Policies (NeurIPS 2021 · similarity 0.89)
- Hierarchical Programmatic Option Framework (NeurIPS 2024 · similarity 0.87)
- Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search (ICLR 2025 · similarity 0.86)
- Hierarchical Reinforcement Learning with Targeted Causal Interventions (ICML 2025 · similarity 0.86)
- Reclaiming the Source of Programmatic Policies: Programmatic versus Latent Spaces (ICLR 2024 · similarity 0.85)
- Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.