Research map / Reinforcement learning / Visual and representation RL
Learning Planning Abstractions from Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PARL, which uses LLM-parsed language annotations of demonstrations to discover discrete verb/object action concepts, then learns a latent object-centric state abstraction, abstract transition and feasibility models for BFS-based planning with low-level policies.
Problem
Learning state and action abstractions for long-horizon, compositional sequential decision-making so agents can plan in an abstract space and generalize to new object counts, concept compositions, and longer horizons, without manually defined symbols or predicates.
Approach
Discover a symbolic action space (verb schemas with object-level arguments) from language-annotated demonstrations via LLM parsing, then learn an object-centric latent transition and feasibility model over these abstract actions and plan with beam-style tree search, rather than learning a flat policy.
Key results
- On BabyAI longer-steps Two-Corridors, Ours (high only) reaches 0.94 success vs 0.44 for High-Level A2C and 0.00 for End-to-End BC/A2C.
- On Kitchen-Worlds Sink with novel environments, Ours achieves 0.61 success vs 0.39 for Goal-Conditioned BC.
- On Kitchen-Worlds novel concept combinations (All-Objects), Ours achieves 0.57 vs 0.14 for Goal-Conditioned BC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PARL high-level planning and learned low-level control instead of end-to-end behavior cloning or A2C | BabyAI · Longer-step Key-Door, All-Objects, and Two-Corridors; novel-combination All-Objects | success rate: 0.00 -> 0.45; 0.00 -> 0.39; 0.00 -> 0.52; 0.00 -> 0.27 | Helped |
| Use PARL high-level planning with oracle low-level policies instead of High-Level BC or A2C | BabyAI · Longer-step Key-Door, All-Objects, and Two-Corridors; novel-combination All-Objects | success rate: 0.27 -> 0.87; 0.00 -> 0.89; 0.44 -> 0.94; 0.00 -> 0.91 | Helped |
| Use PARL rather than Goal-Conditioned BC | Kitchen-Worlds · All-Objects (novel environments), Sink (novel environments), and All-Objects (novel concept combinations) | success rate: 0.82 -> 0.86; 0.39 -> 0.61; 0.14 -> 0.57 | Helped |
| Remove replanning (ablation) | PARL high-only · BabyAI All-Objects (longer steps) | success rate: 0.89 -> 0.65 | Hurt |
| Replace FiLM with an MLP to fuse language and observation (ablation) | PARL high-only · BabyAI All-Objects (longer steps) | success rate: 0.89 -> zero | Hurt |
| Replace the object-centric Transformer with a scene-level FiLM network (ablation) | PARL · Kitchen-Worlds Sink | success rate: 6% decrease | Hurt |
Surprising findings
- Replacing FiLM with an MLP for fusing language and observation drops BabyAI All-Objects longer-step success from 0.89 to zero.
- End-to-end A2C with self-exploration generalizes better than end-to-end BC, while end-to-end RL fails entirely on the longer-horizon tasks in BabyAI.
Limitations
- Requires segmented trajectories aligned with language actions and instance segmentation provided for 3D scenes.
- Assumes language instructions parse into a fixed symbolic form and are complete.
- Object recognition relies on learned concepts without pretrained foundation models for unseen object concepts.
Experimental setup
Datasets: BabyAI demonstrations (bot-generated) Kitchen-Worlds interaction trajectories (PyBullet)
Benchmarks: BabyAI Key-Door BabyAI All-Objects BabyAI Two-Corridors Kitchen-Worlds All-Objects Kitchen-Worlds Sink
Metrics: success rate feasibility F1
Baselines: End-to-End BC End-to-End A2C High-Level BC High-Level A2C Goal-Conditioned BC
Builds on: FiLM Point Cloud Transformer hindsight goal relabeling hierarchical reinforcement learning options
Closest papers by meaning
- Learning with Language-Guided State Abstractions (ICLR 2024 · similarity 0.88)
- Learning Planning Domains from Non-redundant Fully-Observed Traces: Theoretical Foundations and Complexity Analysis (AAAI 2024 · similarity 0.88)
- Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning (AAAI 2025 · similarity 0.87)
- VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning (ICLR 2025 · similarity 0.87)
- Learning Neuro-Symbolic Abstractions for Robot Planning and Learning (AAAI 2024 · similarity 0.87)
- VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.