Research map / Vision-language and multimodal / Robot action and navigation
FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FLIP, a flow-centric interactive world model (CVAE flow generator, flow-conditioned DiT video generator, fine-tuned LIV value module) that plans with tree/beam search over flows and videos using only language-annotated videos.
Problem
Scalable model-based planning for general-purpose robot manipulation from only language and vision inputs, where existing world models either need extra action labels or use action representations too coarse for subtle movements.
Approach
Use dense image flow (query-point trajectories tracked by CoTracker) as a general action representation; plan via beam search where a flow CVAE proposes actions, a flow-conditioned latent video diffusion model simulates short horizons, and a clip-level fine-tuned LIV value function scores states.
Key results
- Model-based planning success on LIBERO-LONG reaches 100% for FLIP vs 2% for UniPi; FMB-M 78% vs 0%.
- Long-horizon video generation FVD on LIBERO-LONG is 35.62 vs 206.28 for IRASim and 610.98 for LVDM.
- Real-world tea scooping success 55% vs 25% for ATM and 15% for Diffusion Policy; cloth unfolding 50% vs 0% for both baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove value-module guidance from beam search (ablation) | FLIP · LIBERO-LONG, FMB-S, FMB-M, and Unfolding | success rate: 78% -> 100%; 52% -> 86%; 40% -> 78%; 70% -> 90% | Helped |
| remove value-module guidance from beam search (ablation) | FLIP · Folding | success rate: 100% -> 100% | No clear effect |
| predict absolute flow coordinates rather than delta scale and direction (ablation) | FLIP action module · LIBERO-LONG and Bridge-V2 | ADE: 12.7 -> 20.5; 11.9 -> 17.9 | Hurt |
| remove auxiliary flow and image reconstruction losses (ablation) | FLIP action module · LIBERO-LONG and Bridge-V2 | ADE: 12.7 -> 14.5; 11.9 -> 12.7 | Hurt |
| use a diffusion model instead of a CVAE for flow generation (ablation) | FLIP action module · LIBERO-LONG and Bridge-V2 | ADE: 12.7 -> 13.4; 11.9 -> 10.2 | Mixed |
| replace fine-grained flow cross-attention with AdaLN-Zero for all conditions (ablation) | FLIP dynamics module · LIBERO-10, Language-Table, and Bridge-V2 | video generation quality | Hurt |
| use short-horizon video clips as adjacent states in value-model fine-tuning instead of adjacent frames (ablation) | FLIP value module · Language-Table and cloth folding | value-curve smoothness | Helped |
| use flow guidance in planning rather than no-value beam search (ablation) | FLIP-NV · LIBERO-LONG, FMB-S, FMB-M, and Unfolding | success rate: 78% -> 100%; 52% -> 86%; 40% -> 78%; 70% -> 90% | Helped |
| pretrain the planner on LIBERO-90 before low-level policy learning (ablation) | flow-conditioned low-level policy · LIBERO-LONG | success rate | No clear effect |
| condition the low-level policy on both flow and video plans rather than video alone (ablation) | FLIP low-level policy · LIBERO-LONG | success rate | Helped |
| condition the low-level policy on video plans without flow (ablation) | FLIP low-level policy · LIBERO-LONG | success rate stability | Hurt |
Surprising findings
- Diffusion-based action module performs comparably to CVAE (better on Bridge-V2, worse on LIBERO-LONG), so the action generator choice matters little.
- Data scaling on LIBERO-LONG shows planning success drops to 0% with 10 demos per task and reaches 100% with 50 demos.
Limitations
- Planning is slow because each step requires extensive video generation, restricting it to quasi-static manipulation.
- Does not use physical properties or 3D scene information.
- Video generation is susceptible to visual distractions, causing distortion and grey outputs after several planning steps.
Experimental setup
Datasets: LIBERO-LONG LIBERO-90 FMB Bridge-V2 Language-Table cloth folding/unfolding (self-collected)
Benchmarks: LIBERO-LONG FMB Bridge-V2 Language-Table LIBERO-10 cloth folding cloth unfolding
Metrics: success rate (human-evaluated) Latent L2 FVD PSNR ADE LTDR
Baselines: UniPi FLIP-NV LVDM IRASim ATM Diffusion Policy OpenVLA FLIP-SC
Builds on: LIV DiT CoTracker ATM UniPi
Closest papers by meaning
- Video Language Planning (ICLR 2024 · similarity 0.90)
- Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation (ICLR 2024 · similarity 0.89)
- VideoVLA: Video Generators Can Be Generalizable Robot Manipulators (NeurIPS 2025 · similarity 0.89)
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.89)
- Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control (NeurIPS 2026 · similarity 0.88)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.