Research map / Vision-language and multimodal / Robot action and navigation
Phoenix: A Motion-based Self-Reflection Framework for Fine-grained Robotic Action Correction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Phoenix uses motion instructions as a bridge: an MLLM-based dual-process motion adjustment module feeds a motion-conditioned diffusion policy that performs fine-grained robot action correction, with lifelong learning from interaction.
Problem
Translating MLLM semantic reflection on failures into fine-grained, low-level robotic action correction in contact-rich manipulation, in a generalizable way.
Approach
Decomposes self-correction into coarse motion instructions (37 types, e.g., "move arm right with gripper closed") produced by an MLLM with a fast predictor plus a chain-of-thought correction module, which conditions a motion-conditioned diffusion policy (learnable motion codebook, separate conditioning) so the generalization burden shifts to the MLLM.
Key results
- Phoenix achieves a 57.8% mean success rate across 9 RoboMimic tasks, versus 48.0% for subgoal self-reflection and 46.9% for the motion-conditioned policy.
- In real-world drawer tasks, Phoenix reaches 75% in-distribution and 55% under pose disruption, outperforming OpenVLA (55%, 30%).
- Lifelong learning raises the motion-conditioned in-distribution real-world success from 60% to 75% after 30 rollouts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace a motion-conditioned policy with Phoenix self-reflection and motion correction | RoboMimic manipulation tasks · RoboMimic (9 tasks) | success rate: 46.9% -> 57.8% | Helped |
| replace subgoal self-reflection with Phoenix motion-based self-reflection | RoboMimic manipulation tasks · RoboMimic (9 tasks) | success rate: 48.0% -> 57.8% | Helped |
| co-train the motion prediction model on expert demonstrations and correction data (ablation) | Motion-conditioned policy · RoboMimic (9 tasks) | success rate: 46.9% -> 49.6% | Helped |
| add self-reflection while co-training on expert demonstrations and correction data (ablation) | Expert-Correction Mixture · RoboMimic (9 tasks) | success rate: 49.6% -> 51.3% | Helped |
| use a unified model for motion prediction and correction instead of the separated motion correction module (ablation) | Phoenix · RoboMimic (9 tasks) | success rate: 57.8% -> 51.3% | Hurt |
| use Phoenix instead of OpenVLA | Real-world drawer open · In-Dis. | success rate: 55% -> 75% | Helped |
| use Phoenix instead of Motion | Real-world drawer open · Pose Dis. | success rate: 35% -> 55% | Helped |
| use Phoenix instead of Motion | Real-world drawer open · Bg. | success rate: 30% -> 45% | Helped |
| use Phoenix instead of Motion | Real-world drawer open · Tex. | success rate: 50% -> 65% | Helped |
| fine-tune the motion prediction model on refined interaction trajectories | Phoenix · RoboMimic lifelong learning | success rate | Helped |
| fine-tune the subgoal self-reflection model on interaction trajectories | Subgoal-based self-reflection · RoboMimic lifelong learning | success rate | No clear effect |
Surprising findings
- Subgoal self-reflection fails to improve lifelong learning during exploration, while motion-level correction does.
- Co-training one unified model for both motion prediction and correction underperforms the separated dual-process design.
Limitations
- Performance depends on the accuracy of motion instructions; fine-grained tasks like Threading are weaker.
- Correction data collection relies on costly human interventions and annotations.
- Chain-of-thought motion correction is slow for real-time use.
Experimental setup
Datasets: RoboMimic expert demonstrations Correction dataset (online interventions, offline annotations, expert demos)
Benchmarks: RoboMimic (9 tasks) real-world drawer open task
Metrics: success rate
Baselines: OpenVLA Task-conditioned policy Subgoal-conditioned policy Motion-conditioned policy Subgoal Self-reflection Human Intervention (Oracle)
Builds on: diffusion policy LLaVA-v1.5 RT-H PaLM-E
Closest papers by meaning
- Chain-of-Correction: Progress-Aware Policy Steering via Anchor-Grounded Predictive Reasoning (NeurIPS 2026 · similarity 0.89)
- Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time Intervention (CVPR 2026 · similarity 0.89)
- Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation (ICLR 2026 · similarity 0.89)
- SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation (ICML 2025 · similarity 0.89)
- ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning (NeurIPS 2026 · similarity 0.88)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.