Research map / Reinforcement learning / Visual and representation RL
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Plan-Seq-Learn uses GPT-4 to produce region/stage plans, a sampling-based motion planner with SAM-based vision to reach each region, and DRQ-v2 RL on local observations to learn contact-rich control for long-horizon robot tasks from visual input.
Problem
Long-horizon robotic manipulation requires both high-level planning and learned low-level contact-rich control; existing LLM planners rely on pre-defined skill libraries, while end-to-end RL struggles with exploration over long horizons.
Approach
Decompose tasks into LLM-generated stages (target region plus stage termination condition), use vision-based motion planning (SAM segmentation, depth point clouds, AIT*) to move the robot near each region, and train a single shared visuomotor RL policy (DRQ-v2) on local wrist-camera observations, with stage termination conditions gating progression.
Key results
- PSL achieves 96% success on RS-NutAssembly (4 stages) vs 23% for SayCan, averaging ~97% on contact-rich nut tasks.
- PSL averages 95% on multi-stage tasks (3-10 stages) vs about 38% for TAMP and 37% for SayCan.
- Wrist-camera PSL converges about 4x faster than fixed-view on RS-Can and chains pre-trained policies up to 5 stages where fixed-view fails.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PSL's language planning, motion-planning sequencing, and online RL rather than the strongest reported baseline | Long-horizon robotics tasks · Robosuite and Kitchen multi-stage tasks | success rate: .23 / .21 -> .96 $\pm$ .08 (RS-NutAssembly); 0.0 / 0.0 -> 1.0 $\pm$ 0.0 (K-MS-10) | Helped |
| Use stage termination checks instead of a 25-step timeout (ablation) | PSL · RS-Can | success rate: 69% -> 100% | Helped |
| Use wrist-camera local observations rather than fixed-view observations (ablation) | PSL · Robosuite Can | learning speed: 4x | Helped |
| Use wrist-camera local observations rather than wrist plus fixed-view observations (ablation) | PSL · Robosuite Can | learning speed: 3x | Helped |
| Use wrist-camera observations rather than fixed-view or wrist-plus-fixed views (ablation) | PSL chaining pre-trained policies · K-MS-5 | success rate: 0.0 $\pm$ 0.0 -> 1.0 $\pm$ 0.0 | Helped |
| Add Gaussian noise to pose estimates (ablation) | PSL · RS-Can, pose noise σ=0.1 | success rate: 1.0 $\pm$ 0.0 -> .75 $\pm$ .07 | Hurt |
| Add Gaussian noise to pose estimates (ablation) | PSL · RS-Can, pose noise σ=0.5 | success rate: 1.0 $\pm$ 0.0 -> 0.0 $\pm$ 0.0 | Hurt |
| Add local observations to E2E or RAPS (ablation) | E2E and RAPS · RS-Lift, RS-Door, RS-Can, RS-NutRound | sample efficiency | No clear effect |
Surprising findings
- Fixed-view PSL fails to chain pre-trained policies because the scene changes, causing out-of-distribution inputs.
- Even with an incorrect LLM plan, PSL can still learn tasks, reaching performance close to E2E in some cases.
Limitations
- Imposes a fixed plan structure (go to region, interact), limiting tasks needing simultaneous multi-object interaction.
- Motion planning makes dynamic tasks challenging.
- The learning module cannot recover if the high-level plan or sequencing module is catastrophically wrong.
Experimental setup
Datasets: Meta-World Obstructed Suite Kitchen Robosuite
Benchmarks: Robosuite Meta-World Kitchen ObstructedSuite
Metrics: task success rate sample efficiency (success vs. trials)
Baselines: E2E (DRQ-v2) RAPS MoPA-RL TAMP (PDDLStream) SayCan
Builds on: DRQ-v2 SayCan AIT* Segment Anything PDDLStream
Closest papers by meaning
- RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (NeurIPS 2025 · similarity 0.89)
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (AAAI 2025 · similarity 0.89)
- Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments (NeurIPS 2024 · similarity 0.89)
- Progress Reward Model for Reinforcement Learning via Large Language Models (NeurIPS 2025 · similarity 0.89)
- Video Language Planning (ICLR 2024 · similarity 0.89)
- SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.