Research map / Reinforcement learning / Visual and representation RL
Efficient Robotic Policy Learning via Latent Space Backward Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Latent Backward Planning (LBP), which predicts a final latent goal from the current observation and language, then recursively generates subgoals backward toward the current state, and conditions a diffusion-based visuomotor policy on them via cross-attention fusion.
Problem
Video and forward subgoal planners for long-horizon robotic manipulation are computationally expensive and accumulate errors, causing off-task predictions that mislead action extraction.
Approach
Plan in latent space by grounding the task into a final latent goal and recursively predicting subgoals backward (coarse-to-fine) from that goal toward the current state, using a single shared MLP subgoal predictor and a Perceiver-style cross-attention fusion of goal and subgoal tokens for the policy.
Key results
- On LIBERO-LONG, LBP with DecisionNCE latent space reaches 88.6% average success, versus 78.6% for Seer and 76.3% for SuSIE.
- On real-robot Shift cups, LBP achieves 67.1 average score versus 41.8 for LCBC and 17.7 for SuSIE.
- Ablation: removing the planner (LCBC) drops LIBERO-LONG success from 88.6% to 77.3%; replacing goal-fusion with average pooling drops it to 79.0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the final latent goal z_g as policy context (ablation) | LCBC on LIBERO-LONG · LIBERO-LONG | Avg. Success: 77.3 -> 83.3 | Helped |
| add intermediate subgoals w_1 and w_2 alongside z_g (ablation) | LBP on LIBERO-LONG · LIBERO-LONG | Avg. Success: 83.3 -> 88.6 | Helped |
| add a third intermediate subgoal w_3 (ablation) | LBP with λ=0.5 on LIBERO-LONG · LIBERO-LONG | Avg. Success: 88.6 -> 83.0 | Hurt |
| change λ from 0.5 to 0.75 with goal and two intermediate subgoals (ablation) | LBP on LIBERO-LONG · LIBERO-LONG | Avg. Success: 88.6 -> 85.0 | Hurt |
| remove the LBP planner (ablation) | LBP on LIBERO-LONG · LIBERO-LONG | Avg. Success: 88.6 -> 77.3 | Hurt |
| replace goal-fusion attention with average pooling (ablation) | LBP on LIBERO-LONG · LIBERO-LONG | Avg. Success: 88.6 -> 79.0 | Hurt |
| use LBP rather than GLCBC | Real-world Move cups task · Move cups | Avg. Score: 62.9 -> 77.9 | Helped |
| use LBP rather than LCBC | Real-world Stack 4 cups task · Stack 4 cups | Avg. Score: 55.0 -> 72.5 | Helped |
| use LBP rather than LCBC | Real-world Shift cups task · Shift cups | Avg. Score: 41.8 -> 67.1 | Helped |
Surprising findings
- Predicting more subgoals is not better: adding a third subgoal (lambda=0.5) lowered success from 88.6% to 83.0%.
Limitations
- Subgoal selection is fixed to a three-step scheme with a hand-chosen lambda rather than learned keyframe selection.
- Real-world evaluation uses only four tasks with 200 demonstrations each and a single robot platform.
- Efficiency claims are not quantified with inference latency measurements.
Experimental setup
Datasets: LIBERO-LONG real-robot AIRBOT demonstrations
Benchmarks: LIBERO-LONG real-robot long-horizon tasks (Stack 3 cups, Move cups, Stack 4 cups, Shift cups)
Metrics: average success rate stage-based average score subgoal MSE
Baselines: MTACT MVP MPI OpenVLA SuSIE Seer LCBC GLCBC
Builds on: SuSIE DecisionNCE Seer Perceiver FiLM
Closest papers by meaning
- STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning (ICLR 2025 · similarity 0.89)
- LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning (NeurIPS 2026 · similarity 0.88)
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.88)
- PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking (NeurIPS 2026 · similarity 0.88)
- Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks (ICLR 2024 · similarity 0.88)
- Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.