Research map / Reinforcement learning / Visual and representation RL

Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Dexterous manipulation transfer · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GTP-FA, a closed-loop grasp-then-plan framework that learns a failure-attribution model (with grasp-conditioned neighbor prior) to route optimization to the grasp or planning module, improving PPO/BC/DP/SAC/pi0.5 on ManiSkill3 and real Franka tasks.

Problem

Grasp and motion planning are tightly coupled, so failures are hard to attribute to grasping versus downstream planning/control, leading to misdirected training and inefficient trial-and-error in long-horizon manipulation.

Approach

A failure-attribution discriminator D over failure modes (FM-G1, FM-G2, FM-P), fused with a grasp-conditioned embedding neighborhood prior E, routes updates: grasp-side risk-penalized, task-prior-aware grasp scoring vs. planning-side hard-start resampling and responsibility-weighted policy training.

Key results

• Full GTP-FA improves average terminal success by +31.3 pp (PPO), +54.0 pp (BC), +25.7 pp (DP), +7.7 pp (SAC), and +17.8 pp (pi0.5) over the original policies in simulation.
• On real Franka tasks, GTP-FA-pi0.5 raises average success from 11.2% to 76.8% (+65.6 pp) over 50 trials per task.
• Naive joint optimization without attribution (11) decreases PPO by 6.0 pp and SAC by 8.2 pp, while planning-only (01) and grasp-only (10) optimization often degrade performance.

Surprising findings

• Planning-side-only optimization drops performance for PPO, SAC and pi0.5 (e.g., SAC -54.5 pp average), showing that naive downstream updates can contaminate learning.

Limitations

• Attribution labels are weak pseudo-labels from repeated trials, not ground-truth causal annotations.
• Evaluated on a limited set of simulated tasks and five real-robot tasks; extension to more objects, platforms and multi-stage tasks is future work.
• Pipeline depends on a VLM prior and GraspNet candidates; per-learner gains vary widely (SAC only +7.7 pp).

Experimental setup

Datasets: ManiSkill3 tasks 100 converted expert trajectories (sim) 300 real-robot expert trajectories

Benchmarks: ManiSkill3 (StackCube, PokeCube, LiftPegUpright, PlaceSphere, PickCube, PushCube, PullCube, PullCubeTool) real-robot Franka tasks (5)

Metrics: success_once success_at_end task success rate

Baselines: PPO BC DP SAC pi0.5 naive joint optimization (11) grasp-side-only (10) planning-side-only (01)

Builds on: GraspNet pi0.5 PPO SAC

Closest papers by meaning

• AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation (ICLR 2025 · similarity 0.87)
• Self-Correcting Robot Manipulation via Gaussian-Splatted Foresight (AAAI 2025 · similarity 0.86)
• Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge (CVPR 2024 · similarity 0.86)
• Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation (NeurIPS 2025 · similarity 0.85)
• Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.85)
• SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation (ICML 2025 · similarity 0.85)